October 19, 2019

3284 words 16 mins read

Paper Group ANR 264

Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision. Multi-Level Factorisation Net for Person Re-Identification. Intrinsic dimension and its application to association rules. Automatic segmentation of the Foveal Avascular Zone in ophthalmological OCT-A images. Consistent estimation of the max-flow problem: Towards unsupervised i …

Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision


Title	Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision
Authors	Hai Wang, Hoifung Poon
Abstract	Deep learning has emerged as a versatile tool for a wide range of NLP tasks, due to its superior capacity in representation learning. But its applicability is limited by the reliance on annotated examples, which are difficult to produce at scale. Indirect supervision has emerged as a promising direction to address this bottleneck, either by introducing labeling functions to automatically generate noisy examples from unlabeled text, or by imposing constraints over interdependent label decisions. A plethora of methods have been proposed, each with respective strengths and limitations. Probabilistic logic offers a unifying language to represent indirect supervision, but end-to-end modeling with probabilistic logic is often infeasible due to intractable inference and learning. In this paper, we propose deep probabilistic logic (DPL) as a general framework for indirect supervision, by composing probabilistic logic with deep learning. DPL models label decisions as latent variables, represents prior knowledge on their relations using weighted first-order logical formulas, and alternates between learning a deep neural network for the end task and refining uncertain formula weights for indirect supervision, using variational EM. This framework subsumes prior indirect supervision methods as special cases, and enables novel combination via infusion of rich domain and linguistic knowledge. Experiments on biomedical machine reading demonstrate the promise of this approach.
Tasks	Reading Comprehension, Representation Learning
Published	2018-08-26
URL	http://arxiv.org/abs/1808.08485v1
PDF	http://arxiv.org/pdf/1808.08485v1.pdf
PWC	https://paperswithcode.com/paper/deep-probabilistic-logic-a-unifying-framework
Repo
Framework

Multi-Level Factorisation Net for Person Re-Identification


Title	Multi-Level Factorisation Net for Person Re-Identification
Authors	Xiaobin Chang, Timothy M. Hospedales, Tao Xiang
Abstract	Key to effective person re-identification (Re-ID) is modelling discriminative and view-invariant factors of person appearance at both high and low semantic levels. Recently developed deep Re-ID models either learn a holistic single semantic level feature representation and/or require laborious human annotation of these factors as attributes. We propose Multi-Level Factorisation Net (MLFN), a novel network architecture that factorises the visual appearance of a person into latent discriminative factors at multiple semantic levels without manual annotation. MLFN is composed of multiple stacked blocks. Each block contains multiple factor modules to model latent factors at a specific level, and factor selection modules that dynamically select the factor modules to interpret the content of each input image. The outputs of the factor selection modules also provide a compact latent factor descriptor that is complementary to the conventional deeply learned features. MLFN achieves state-of-the-art results on three Re-ID datasets, as well as compelling results on the general object categorisation CIFAR-100 dataset.
Tasks	Person Re-Identification
Published	2018-03-24
URL	http://arxiv.org/abs/1803.09132v2
PDF	http://arxiv.org/pdf/1803.09132v2.pdf
PWC	https://paperswithcode.com/paper/multi-level-factorisation-net-for-person-re
Repo
Framework

Intrinsic dimension and its application to association rules


Title	Intrinsic dimension and its application to association rules
Authors	Tom Hanika, Friedrich Martin Schneider, Gerd Stumme
Abstract	The curse of dimensionality in the realm of association rules is twofold. Firstly, we have the well known exponential increase in computational complexity with increasing item set size. Secondly, there is a \emph{related curse} concerned with the distribution of (spare) data itself in high dimension. The former problem is often coped with by projection, i.e., feature selection, whereas the best known strategy for the latter is avoidance. This work summarizes the first attempt to provide a computationally feasible method for measuring the extent of dimension curse present in a data set with respect to a particular class machine of learning procedures. This recent development enables the application of various other methods from geometric analysis to be investigated and applied in machine learning procedures in the presence of high dimension.
Tasks	Feature Selection
Published	2018-05-15
URL	http://arxiv.org/abs/1805.05714v1
PDF	http://arxiv.org/pdf/1805.05714v1.pdf
PWC	https://paperswithcode.com/paper/intrinsic-dimension-and-its-application-to
Repo
Framework

Automatic segmentation of the Foveal Avascular Zone in ophthalmological OCT-A images


Title	Automatic segmentation of the Foveal Avascular Zone in ophthalmological OCT-A images
Authors	Macarena Díaz, Jorge Novo, Paula Cutrín, Francisco Gómez-Ulla, Manuel G. Penedo, Marcos Ortega
Abstract	Angiography by Optical Coherence Tomography is a non-invasive retinal imaging modality of recent appearance that allows the visualization of the vascular structure at predefined depths based on the detection of the blood movement. OCT-A images constitute a suitable scenario to analyse the retinal vascular properties of regions of interest, measuring the characteristics of the foveal vascular and avascular zones. Extracted parameters of this region can be used as prognostic factors that determine if the patient suffers from certain pathologies, indicating the associated pathological degree. The manual extraction of these biomedical parameters is a long, tedious and subjective process, introducing a significant intra and inter-expert variability, which penalizes the utility of the measurements. In addition, the absence of tools that automatically facilitate these calculations encourages the creation of computer-aided diagnosis frameworks that ease the doctor’s work, increasing their productivity and making viable the use of this type of vascular biomarkers. We propose a fully automatic system that identifies and precisely segments the region of the foveal avascular zone (FAZ) using a novel ophthalmological image modality as is OCT-A. The system combines different image processing techniques to firstly identify the region where the FAZ is contained and, secondly, proceed with the extraction of its precise contour. The system was validated using a representative set of 168 OCT-A images, providing accurate results with the best correlation with the manual measurements of two experts clinician of 0.93 as well as a Jaccard’s index of 0.82 of the best experimental case. This tool provides an accurate FAZ measurement with the desired objectivity and reproducibility, being very useful for the analysis of relevant vascular diseases through the study of the retinal microcirculation.
Tasks
Published	2018-11-26
URL	http://arxiv.org/abs/1811.10374v1
PDF	http://arxiv.org/pdf/1811.10374v1.pdf
PWC	https://paperswithcode.com/paper/automatic-segmentation-of-the-foveal
Repo
Framework

Consistent estimation of the max-flow problem: Towards unsupervised image segmentation


Title	Consistent estimation of the max-flow problem: Towards unsupervised image segmentation
Authors	Ashif Sikandar Iquebal, Satish Bukkapatnam
Abstract	Advances in the image-based diagnostics of complex biological and manufacturing processes have brought unsupervised image segmentation to the forefront of enabling automated, on the fly decision making. However, most existing unsupervised segmentation approaches are either computationally complex or require manual parameter selection (e.g., flow capacities in max-flow/min-cut segmentation). In this work, we present a fully unsupervised segmentation approach using a continuous max-flow formulation over the image domain while optimally estimating the flow parameters from the image characteristics. More specifically, we show that the maximum a posteriori estimate of the image labels can be formulated as a continuous max-flow problem given the flow capacities are known. The flow capacities are then iteratively obtained by employing a novel Markov random field prior over the image domain. We present theoretical results to establish the posterior consistency of the flow capacities. We compare the performance of our approach on two real-world case studies including brain tumor image segmentation and defect identification in additively manufactured components using electron microscopic images. Comparative results with several state-of-the-art supervised as well as unsupervised methods suggest that the present method performs statistically similar to the supervised methods, but results in more than 90% improvement in the Dice score when compared to the state-of-the-art unsupervised methods.
Tasks	Brain Tumor Segmentation, Decision Making, Semantic Segmentation
Published	2018-11-01
URL	https://arxiv.org/abs/1811.00220v2
PDF	https://arxiv.org/pdf/1811.00220v2.pdf
PWC	https://paperswithcode.com/paper/unsupervised-image-segmentation-via-maximum-a
Repo
Framework

Approximate Query Matching for Image Retrieval


Title	Approximate Query Matching for Image Retrieval
Authors	Abhijit Suprem, Polo Chau
Abstract	Traditional image recognition involves identifying the key object in a portrait-type image with a single object focus (ILSVRC, AlexNet, and VGG). More recent approaches consider dense image recognition - segmenting an image with appropriate bounding boxes and performing image recognition within these bounding boxes (Semantic segmentation). The Visual Genome dataset [5] is an attempt to bridge these various approaches to a cohesive dataset for each subtask - bounding box generation, image recognition, captioning, and a new operation: scene graph generation. Our focus is on using such scene graphs to perform graph search on image databases to holistically retrieve images based on a search criteria. We develop a method to store scene graphs and metadata in graph databases (using Neo4J) and to perform fast approximate retrieval of images based on a graph search query. We process more complex queries than single object search, e.g. “girl eating cake” retrieves images that contain the specified relation as well as variations.
Tasks	Graph Generation, Image Retrieval, Scene Graph Generation, Semantic Segmentation
Published	2018-03-14
URL	http://arxiv.org/abs/1803.05401v1
PDF	http://arxiv.org/pdf/1803.05401v1.pdf
PWC	https://paperswithcode.com/paper/approximate-query-matching-for-image
Repo
Framework

Towards Composable Bias Rating of AI Services


Title	Towards Composable Bias Rating of AI Services
Authors	Biplav Srivastava, Francesca Rossi
Abstract	A new wave of decision-support systems are being built today using AI services that draw insights from data (like text and video) and incorporate them in human-in-the-loop assistance. However, just as we expect humans to be ethical, the same expectation needs to be met by automated systems that increasingly get delegated to act on their behalf. A very important aspect of an ethical behavior is to avoid (intended, perceived, or accidental) bias. Bias occurs when the data distribution is not representative enough of the natural phenomenon one wants to model and reason about. The possibly biased behavior of a service is hard to detect and handle if the AI service is merely being used and not developed from scratch, since the training data set is not available. In this situation, we envisage a 3rd party rating agency that is independent of the API producer or consumer and has its own set of biased and unbiased data, with customizable distributions. We propose a 2-step rating approach that generates bias ratings signifying whether the AI service is unbiased compensating, data-sensitive biased, or biased. The approach also works on composite services. We implement it in the context of text translation and report interesting results.
Tasks
Published	2018-07-31
URL	http://arxiv.org/abs/1808.00089v2
PDF	http://arxiv.org/pdf/1808.00089v2.pdf
PWC	https://paperswithcode.com/paper/towards-composable-bias-rating-of-ai-services
Repo
Framework

Uncertainty-driven Sanity Check: Application to Postoperative Brain Tumor Cavity Segmentation


Title	Uncertainty-driven Sanity Check: Application to Postoperative Brain Tumor Cavity Segmentation
Authors	Alain Jungo, Raphael Meier, Ekin Ermis, Evelyn Herrmann, Mauricio Reyes
Abstract	Uncertainty estimates of modern neuronal networks provide additional information next to the computed predictions and are thus expected to improve the understanding of the underlying model. Reliable uncertainties are particularly interesting for safety-critical computer-assisted applications in medicine, e.g., neurosurgical interventions and radiotherapy planning. We propose an uncertainty-driven sanity check for the identification of segmentation results that need particular expert review. Our method uses a fully-convolutional neural network and computes uncertainty estimates by the principle of Monte Carlo dropout. We evaluate the performance of the proposed method on a clinical dataset with 30 postoperative brain tumor images. The method can segment the highly inhomogeneous resection cavities accurately (Dice coefficients 0.792 $\pm$ 0.154). Furthermore, the proposed sanity check is able to detect the worst segmentation and three out of the four outliers. The results highlight the potential of using the additional information from the model’s parameter uncertainty to validate the segmentation performance of a deep learning model.
Tasks
Published	2018-06-08
URL	http://arxiv.org/abs/1806.03106v1
PDF	http://arxiv.org/pdf/1806.03106v1.pdf
PWC	https://paperswithcode.com/paper/uncertainty-driven-sanity-check-application
Repo
Framework

Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles


Title	Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
Authors	Edward Grefenstette, Robert Stanforth, Brendan O’Donoghue, Jonathan Uesato, Grzegorz Swirszcz, Pushmeet Kohli
Abstract	While deep learning has led to remarkable results on a number of challenging problems, researchers have discovered a vulnerability of neural networks in adversarial settings, where small but carefully chosen perturbations to the input can make the models produce extremely inaccurate outputs. This makes these models particularly unsuitable for safety-critical application domains (e.g. self-driving cars) where robustness is extremely important. Recent work has shown that augmenting training with adversarially generated data provides some degree of robustness against test-time attacks. In this paper we investigate how this approach scales as we increase the computational budget given to the defender. We show that increasing the number of parameters in adversarially-trained models increases their robustness, and in particular that ensembling smaller models while adversarially training the entire ensemble as a single model is a more efficient way of spending said budget than simply using a larger single model. Crucially, we show that it is the adversarial training of the ensemble, rather than the ensembling of adversarially trained models, which provides robustness.
Tasks	Self-Driving Cars
Published	2018-11-22
URL	http://arxiv.org/abs/1811.09300v1
PDF	http://arxiv.org/pdf/1811.09300v1.pdf
PWC	https://paperswithcode.com/paper/strength-in-numbers-trading-off-robustness
Repo
Framework

Computational and informatics advances for reproducible data analysis in neuroimaging


Title	Computational and informatics advances for reproducible data analysis in neuroimaging
Authors	Russell A. Poldrack, Krzysztof J. Gorgolewski, Gael Varoquaux
Abstract	The reproducibility of scientific research has become a point of critical concern. We argue that openness and transparency are critical for reproducibility, and we outline an ecosystem for open and transparent science that has emerged within the human neuroimaging community. We discuss the range of open data sharing resources that have been developed for neuroimaging data, and the role of data standards (particularly the Brain Imaging Data Structure) in enabling the automated sharing, processing, and reuse of large neuroimaging datasets. We outline how the open-source Python language has provided the basis for a data science platform that enables reproducible data analysis and visualization. We also discuss how new advances in software engineering, such as containerization, provide the basis for greater reproducibility in data analysis. The emergence of this new ecosystem provides an example for many areas of science that are currently struggling with reproducibility.
Tasks
Published	2018-09-24
URL	http://arxiv.org/abs/1809.10024v1
PDF	http://arxiv.org/pdf/1809.10024v1.pdf
PWC	https://paperswithcode.com/paper/computational-and-informatics-advances-for
Repo
Framework

Towards learning-to-learn


Title	Towards learning-to-learn
Authors	Benjamin James Lansdell, Konrad Paul Kording
Abstract	In good old-fashioned artificial intelligence (GOFAI), humans specified systems that solved problems. Much of the recent progress in AI has come from replacing human insights by learning. However, learning itself is still usually built by humans – specifically the choice that parameter updates should follow the gradient of a cost function. Yet, in analogy with GOFAI, there is no reason to believe that humans are particularly good at defining such learning systems: we may expect learning itself to be better if we learn it. Recent research in machine learning has started to realize the benefits of that strategy. We should thus expect this to be relevant for neuroscience: how could the correct learning rules be acquired? Indeed, cognitive science has long shown that humans learn-to-learn, which is potentially responsible for their impressive learning abilities. Here we discuss ideas across machine learning, neuroscience, and cognitive science that matter for the principle of learning-to-learn.
Tasks
Published	2018-11-01
URL	http://arxiv.org/abs/1811.00231v3
PDF	http://arxiv.org/pdf/1811.00231v3.pdf
PWC	https://paperswithcode.com/paper/towards-learning-to-learn
Repo
Framework

Statistical Piano Reduction Controlling Performance Difficulty


Title	Statistical Piano Reduction Controlling Performance Difficulty
Authors	Eita Nakamura, Kazuyoshi Yoshii
Abstract	We present a statistical-modelling method for piano reduction, i.e. converting an ensemble score into piano scores, that can control performance difficulty. While previous studies have focused on describing the condition for playable piano scores, it depends on player’s skill and can change continuously with the tempo. We thus computationally quantify performance difficulty as well as musical fidelity to the original score, and formulate the problem as optimization of musical fidelity under constraints on difficulty values. First, performance difficulty measures are developed by means of probabilistic generative models for piano scores and the relation to the rate of performance errors is studied. Second, to describe musical fidelity, we construct a probabilistic model integrating a prior piano-score model and a model representing how ensemble scores are likely to be edited. An iterative optimization algorithm for piano reduction is developed based on statistical inference of the model. We confirm the effect of the iterative procedure; we find that subjective difficulty and musical fidelity monotonically increase with controlled difficulty values; and we show that incorporating sequential dependence of pitches and fingering motion in the piano-score model improves the quality of reduction scores in high-difficulty cases.
Tasks
Published	2018-08-15
URL	http://arxiv.org/abs/1808.05006v2
PDF	http://arxiv.org/pdf/1808.05006v2.pdf
PWC	https://paperswithcode.com/paper/statistical-piano-reduction-controlling
Repo
Framework

Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods


Title	Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods
Authors	Laurence Aitchison
Abstract	Neural network optimization methods fall into two broad classes: adaptive methods such as Adam and non-adaptive methods such as vanilla stochastic gradient descent (SGD). Here, we formulate the problem of neural network optimization as Bayesian filtering. We find that state-of-the-art adaptive (AdamW) and non-adaptive (SGD) methods can be recovered by taking limits as the amount of information about the parameter gets large or small, respectively. As such, we develop a new neural network optimization algorithm, AdaBayes, that adaptively transitions between SGD-like and Adam(W)-like behaviour. This algorithm converges more rapidly than Adam in the early part of learning, and has generalisation performance competitive with SGD.
Tasks
Published	2018-07-19
URL	https://arxiv.org/abs/1807.07540v4
PDF	https://arxiv.org/pdf/1807.07540v4.pdf
PWC	https://paperswithcode.com/paper/a-unified-theory-of-adaptive-stochastic
Repo
Framework

Neural Phrase-to-Phrase Machine Translation


Title	Neural Phrase-to-Phrase Machine Translation
Authors	Jiangtao Feng, Lingpeng Kong, Po-Sen Huang, Chong Wang, Da Huang, Jiayuan Mao, Kan Qiao, Dengyong Zhou
Abstract	In this paper, we propose Neural Phrase-to-Phrase Machine Translation (NP$^2$MT). Our model uses a phrase attention mechanism to discover relevant input (source) segments that are used by a decoder to generate output (target) phrases. We also design an efficient dynamic programming algorithm to decode segments that allows the model to be trained faster than the existing neural phrase-based machine translation method by Huang et al. (2018). Furthermore, our method can naturally integrate with external phrase dictionaries during decoding. Empirical experiments show that our method achieves comparable performance with the state-of-the art methods on benchmark datasets. However, when the training and testing data are from different distributions or domains, our method performs better.
Tasks	Machine Translation
Published	2018-11-06
URL	http://arxiv.org/abs/1811.02172v1
PDF	http://arxiv.org/pdf/1811.02172v1.pdf
PWC	https://paperswithcode.com/paper/neural-phrase-to-phrase-machine-translation
Repo
Framework

DeepVesselNet: Vessel Segmentation, Centerline Prediction, and Bifurcation Detection in 3-D Angiographic Volumes


Title	DeepVesselNet: Vessel Segmentation, Centerline Prediction, and Bifurcation Detection in 3-D Angiographic Volumes
Authors	Giles Tetteh, Velizar Efremov, Nils D. Forkert, Matthias Schneider, Jan Kirschke, Bruno Weber, Claus Zimmer, Marie Piraud, Bjoern H. Menze
Abstract	We present DeepVesselNet, an architecture tailored to the challenges faced when extracting vessel networks or trees and corresponding features in 3-D angiographic volumes using deep learning. We discuss the problems of low execution speed and high memory requirements associated with full 3-D convolutional networks, high-class imbalance arising from the low percentage of vessel voxels, and unavailability of accurately annotated training data - and offer solutions as the building blocks of DeepVesselNet. First, we formulate 2-D orthogonal cross-hair filters which make use of 3-D context information at a reduced computational burden. Second, we introduce a class balancing cross-entropy loss function with false positive rate correction to handle the high-class imbalance and high false positive rate problems associated with existing loss functions. Finally, we generate synthetic dataset using a computational angiogenesis model capable of generating vascular trees under physiological constraints on local network structure and topology and use these data for transfer learning. DeepVesselNet is optimized for segmenting and analyzing vessels, and we test the performance on a range of angiographic volumes including clinical MRA data of the human brain, as well as X-ray tomographic microscopy scans of the rat brain. Our experiments show that, by replacing 3-D filters with cross-hair filters in our network, we achieve over 23% improvement in speed, lower memory footprint, lower network complexity which prevents overfitting and comparable accuracy (with a Cox-Wilcoxon paired sample significance test p-value of 0.07 when compared to full 3-D filters). Our class balancing metric is crucial for training the network and transfer learning with synthetic data is an efficient, robust, and very generalizable approach leading to a network that excels in a variety of angiography segmentation tasks.
Tasks	Transfer Learning
Published	2018-03-25
URL	https://arxiv.org/abs/1803.09340v3
PDF	https://arxiv.org/pdf/1803.09340v3.pdf
PWC	https://paperswithcode.com/paper/deepvesselnet-vessel-segmentation-centerline
Repo
Framework