April 2, 2020

3557 words 17 mins read

Paper Group ANR 387

Probabilistic 3D Multilabel Real-time Mapping for Multi-object Manipulation. Effects of annotation granularity in deep learning models for histopathological images. Generating Scientific Question Answering Corpora from Q&A forums. Data Vision: Learning to See Through Algorithmic Abstraction. Unsupervised Semantic Attribute Discovery and Control in …

Probabilistic 3D Multilabel Real-time Mapping for Multi-object Manipulation


Title	Probabilistic 3D Multilabel Real-time Mapping for Multi-object Manipulation
Authors	Kentaro Wada, Kei Okada, Masayuki Inaba
Abstract	Probabilistic 3D map has been applied to object segmentation with multiple camera viewpoints, however, conventional methods lack of real-time efficiency and functionality of multilabel object mapping. In this paper, we propose a method to generate three-dimensional map with multilabel occupancy in real-time. Extending our previous work in which only target label occupancy is mapped, we achieve multilabel object segmentation in a single looking around action. We evaluate our method by testing segmentation accuracy with 39 different objects, and applying it to a manipulation task of multiple objects in the experiments. Our mapping-based method outperforms the conventional projection-based method by 40 - 96% relative (12.6 mean $IU_{3d}$), and robot successfully recognizes (86.9%) and manipulates multiple objects (60.7%) in an environment with heavy occlusions.
Tasks	Semantic Segmentation
Published	2020-01-16
URL	https://arxiv.org/abs/2001.05752v1
PDF	https://arxiv.org/pdf/2001.05752v1.pdf
PWC	https://paperswithcode.com/paper/probabilistic-3d-multilabel-real-time-mapping
Repo
Framework

Effects of annotation granularity in deep learning models for histopathological images


Title	Effects of annotation granularity in deep learning models for histopathological images
Authors	Jiangbo Shi, Zeyu Gao, Haichuan Zhang, Pargorn Puttapirat, Chunbao Wang, Xiangrong Zhang, Chen Li
Abstract	Pathological is crucial to cancer diagnosis. Usually, Pathologists draw their conclusion based on observed cell and tissue structure on histology slides. Rapid development in machine learning, especially deep learning have established robust and accurate classifiers. They are being used to analyze histopathological slides and assist pathologists in diagnosis. Most machine learning systems rely heavily on annotated data sets to gain experiences and knowledge to correctly and accurately perform various tasks such as classification and segmentation. This work investigates different granularity of annotations in histopathological data set including image-wise, bounding box, ellipse-wise, and pixel-wise to verify the influence of annotation in pathological slide on deep learning models. We design corresponding experiments to test classification and segmentation performance of deep learning models based on annotations with different annotation granularity. In classification, state-of-the-art deep learning-based classifiers perform better when trained by pixel-wise annotation dataset. On average, precision, recall and F1-score improves by 7.87%, 8.83% and 7.85% respectively. Thus, it is suggested that finer granularity annotations are better utilized by deep learning algorithms in classification tasks. Similarly, semantic segmentation algorithms can achieve 8.33% better segmentation accuracy when trained by pixel-wise annotations. Our study shows not only that finer-grained annotation can improve the performance of deep learning models, but also help extracts more accurate phenotypic information from histopathological slides. Intelligence systems trained on granular annotations may help pathologists inspecting certain regions for better diagnosis. The compartmentalized prediction approach similar to this work may contribute to phenotype and genotype association studies.
Tasks	Semantic Segmentation
Published	2020-01-14
URL	https://arxiv.org/abs/2001.04663v1
PDF	https://arxiv.org/pdf/2001.04663v1.pdf
PWC	https://paperswithcode.com/paper/effects-of-annotation-granularity-in-deep
Repo
Framework

Generating Scientific Question Answering Corpora from Q&A forums


Title	Generating Scientific Question Answering Corpora from Q&A forums
Authors	Andre Lamurias, Diana Sousa, Francisco M. Couto
Abstract	Question Answering (QA) is a natural language processing task that aims at retrieving relevant answers to user questions. While much progress has been made in this area, biomedical questions are still a challenge to most QA approaches, due to the complexity of the domain and limited availability of training sets. We present a method to automatically extract question-article pairs from Q&A web forums, which can be used for document retrieval and QA tasks. The proposed framework extracts questions from selected forums as well as answers that contain citations that can be mapped to a unique entry of a digital library. This way, QA systems based on document retrieval can be developed and evaluated using the question-article pairs annotated by users of these forums. We generated the SciQA corpus by applying our framework to three forums, obtaining 5,432 questions and 10,208 question-article pairs. We evaluated how the number of articles associated with each question and the number of votes on each answer affects the performance of baseline document retrieval approaches. Also, we trained a state-of-the-art deep learning model that obtained higher scores in most test batches than a model trained only on a dataset manually annotated by experts. The framework described in this paper can be used to update the SciQA corpus from the same forums as new posts are made, and from other forums that support their answers with documents.
Tasks	Question Answering
Published	2020-02-06
URL	https://arxiv.org/abs/2002.02375v1
PDF	https://arxiv.org/pdf/2002.02375v1.pdf
PWC	https://paperswithcode.com/paper/generating-scientific-question-answering
Repo
Framework

Data Vision: Learning to See Through Algorithmic Abstraction


Title	Data Vision: Learning to See Through Algorithmic Abstraction
Authors	Samir Passi, Steven J. Jackson
Abstract	Learning to see through data is central to contemporary forms of algorithmic knowledge production. While often represented as a mechanical application of rules, making algorithms work with data requires a great deal of situated work. This paper examines how the often-divergent demands of mechanization and discretion manifest in data analytic learning environments. Drawing on research in CSCW and the social sciences, and ethnographic fieldwork in two data learning environments, we show how an algorithm’s application is seen sometimes as a mechanical sequence of rules and at other times as an array of situated decisions. Casting data analytics as a rule-based (rather than rule-bound) practice, we show that effective data vision requires would-be analysts to straddle the competing demands of formal abstraction and empirical contingency. We conclude by discussing how the notion of data vision can help better leverage the role of human work in data analytic learning, research, and practice.
Tasks
Published	2020-02-09
URL	https://arxiv.org/abs/2002.03387v1
PDF	https://arxiv.org/pdf/2002.03387v1.pdf
PWC	https://paperswithcode.com/paper/data-vision-learning-to-see-through
Repo
Framework

Unsupervised Semantic Attribute Discovery and Control in Generative Models


Title	Unsupervised Semantic Attribute Discovery and Control in Generative Models
Authors	William Paul, I-Jeng Wang, Fady Alajaji, Philippe Burlina
Abstract	This work focuses on the ability to control via latent space factors semantic image attributes in generative models, and the faculty to discover mappings from factors to attributes in an unsupervised fashion. The discovery of controllable semantic attributes is of special importance, as it would facilitate higher level tasks such as unsupervised representation learning to improve anomaly detection, or the controlled generation of novel data for domain shift and imbalanced datasets. The ability to control semantic attributes is related to the disentanglement of latent factors, which dictates that latent factors be “uncorrelated” in their effects. Unfortunately, despite past progress, the connection between control and disentanglement remains, at best, confused and entangled, requiring clarifications we hope to provide in this work. To this end, we study the design of algorithms for image generation that allow unsupervised discovery and control of semantic attributes.We make several contributions: a) We bring order to the concepts of control and disentanglement, by providing an analytical derivation that connects mutual information maximization, which promotes attribute control, to total correlation minimization, which relates to disentanglement. b) We propose hybrid generative model architectures that use mutual information maximization with multi-scale style transfer. c) We introduce a novel metric to characterize the performance of semantic attributes control. We report experiments that appear to demonstrate, quantitatively and qualitatively, the ability of the proposed model to perform satisfactory control while still preserving competitive visual quality. We compare to other state of the art methods (e.g., Frechet inception distance (FID)= 9.90 on CelebA and 4.52 on EyePACS).
Tasks	Anomaly Detection, Image Generation, Representation Learning, Style Transfer, Unsupervised Representation Learning
Published	2020-02-25
URL	https://arxiv.org/abs/2002.11169v1
PDF	https://arxiv.org/pdf/2002.11169v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-semantic-attribute-discovery-and
Repo
Framework

Learning Cross-Context Entity Representations from Text


Title	Learning Cross-Context Entity Representations from Text
Authors	Jeffrey Ling, Nicholas FitzGerald, Zifei Shan, Livio Baldini Soares, Thibault Févry, David Weiss, Tom Kwiatkowski
Abstract	Language modeling tasks, in which words, or word-pieces, are predicted on the basis of a local context, have been very effective for learning word embeddings and context dependent representations of phrases. Motivated by the observation that efforts to code world knowledge into machine readable knowledge bases or human readable encyclopedias tend to be entity-centric, we investigate the use of a fill-in-the-blank task to learn context independent representations of entities from the text contexts in which those entities were mentioned. We show that large scale training of neural models allows us to learn high quality entity representations, and we demonstrate successful results on four domains: (1) existing entity-level typing benchmarks, including a 64% error reduction over previous work on TypeNet (Murty et al., 2018); (2) a novel few-shot category reconstruction task; (3) existing entity linking benchmarks, where we match the state-of-the-art on CoNLL-Aida without linking-specific features and obtain a score of 89.8% on TAC-KBP 2010 without using any alias table, external knowledge base or in domain training data and (4) answering trivia questions, which uniquely identify entities. Our global entity representations encode fine-grained type categories, such as Scottish footballers, and can answer trivia questions such as: Who was the last inmate of Spandau jail in Berlin?
Tasks	Entity Linking, Language Modelling, Learning Word Embeddings, Word Embeddings
Published	2020-01-11
URL	https://arxiv.org/abs/2001.03765v1
PDF	https://arxiv.org/pdf/2001.03765v1.pdf
PWC	https://paperswithcode.com/paper/learning-cross-context-entity-representations-1
Repo
Framework

Leveraging Rationales to Improve Human Task Performance


Title	Leveraging Rationales to Improve Human Task Performance
Authors	Devleena Das, Sonia Chernova
Abstract	Machine learning (ML) systems across many application areas are increasingly demonstrating performance that is beyond that of humans. In response to the proliferation of such models, the field of Explainable AI (XAI) has sought to develop techniques that enhance the transparency and interpretability of machine learning methods. In this work, we consider a question not previously explored within the XAI and ML communities: Given a computational system whose performance exceeds that of its human user, can explainable AI capabilities be leveraged to improve the performance of the human? We study this question in the context of the game of Chess, for which computational game engines that surpass the performance of the average player are widely available. We introduce the Rationale-Generating Algorithm, an automated technique for generating rationales for utility-based computational methods, which we evaluate with a multi-day user study against two baselines. The results show that our approach produces rationales that lead to statistically significant improvement in human task performance, demonstrating that rationales automatically generated from an AI’s internal task model can be used not only to explain what the system is doing, but also to instruct the user and ultimately improve their task performance.
Tasks	Game of Chess
Published	2020-02-11
URL	https://arxiv.org/abs/2002.04202v1
PDF	https://arxiv.org/pdf/2002.04202v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-rationales-to-improve-human-task
Repo
Framework

Learning with Differentiable Perturbed Optimizers


Title	Learning with Differentiable Perturbed Optimizers
Authors	Quentin Berthet, Mathieu Blondel, Olivier Teboul, Marco Cuturi, Jean-Philippe Vert, Francis Bach
Abstract	Machine learning pipelines often rely on optimization procedures to make discrete decisions (e.g. sorting, picking closest neighbors, finding shortest paths or optimal matchings). Although these discrete decisions are easily computed in a forward manner, they cannot be used to modify model parameters using first-order optimization techniques because they break the back-propagation of computational graphs. In order to expand the scope of learning problems that can be solved in an end-to-end fashion, we propose a systematic method to transform a block that outputs an optimal discrete decision into a differentiable operation. Our approach relies on stochastic perturbations of these parameters, and can be used readily within existing solvers without the need for ad hoc regularization or smoothing. These perturbed optimizers yield solutions that are differentiable and never locally constant. The amount of smoothness can be tuned via the chosen noise amplitude, whose impact we analyze. The derivatives of these perturbed solvers can be evaluated efficiently. We also show how this framework can be connected to a family of losses developed in structured prediction, and describe how these can be used in unsupervised and supervised learning, with theoretical guarantees. We demonstrate the performance of our approach on several machine learning tasks in experiments on synthetic and real data.
Tasks	Structured Prediction
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08676v1
PDF	https://arxiv.org/pdf/2002.08676v1.pdf
PWC	https://paperswithcode.com/paper/learning-with-differentiable-perturbed
Repo
Framework

Two Huge Title and Keyword Generation Corpora of Research Articles


Title	Two Huge Title and Keyword Generation Corpora of Research Articles
Authors	Erion Çano, Ondřej Bojar
Abstract	Recent developments in sequence-to-sequence learning with neural networks have considerably improved the quality of automatically generated text summaries and document keywords, stipulating the need for even bigger training corpora. Metadata of research articles are usually easy to find online and can be used to perform research on various tasks. In this paper, we introduce two huge datasets for text summarization (OAGSX) and keyword generation (OAGKX) research, containing 34 million and 23 million records, respectively. The data were retrieved from the Open Academic Graph which is a network of research profiles and publications. We carefully processed each record and also tried several extractive and abstractive methods of both tasks to create performance baselines for other researchers. We further illustrate the performance of those methods previewing their outputs. In the near future, we would like to apply topic modeling on the two sets to derive subsets of research articles from more specific disciplines.
Tasks	Text Summarization
Published	2020-02-11
URL	https://arxiv.org/abs/2002.04689v1
PDF	https://arxiv.org/pdf/2002.04689v1.pdf
PWC	https://paperswithcode.com/paper/two-huge-title-and-keyword-generation-corpora
Repo
Framework

An Ontology-Aware Framework for Audio Event Classification


Title	An Ontology-Aware Framework for Audio Event Classification
Authors	Yiwei Sun, Shabnam Ghaffarzadegan
Abstract	Recent advancements in audio event classification often ignore the structure and relation between the label classes available as prior information. This structure can be defined by ontology and augmented in the classifier as a form of domain knowledge. To capture such dependencies between the labels, we propose an ontology-aware neural network containing two components: feed-forward ontology layers and graph convolutional networks (GCN). The feed-forward ontology layers capture the intra-dependencies of labels between different levels of ontology. On the other hand, GCN mainly models inter-dependency structure of labels within an ontology level. The framework is evaluated on two benchmark datasets for single-label and multi-label audio event classification tasks. The results demonstrate the proposed solutions efficacy to capture and explore the ontology relations and improve the classification performance.
Tasks
Published	2020-01-27
URL	https://arxiv.org/abs/2001.10048v1
PDF	https://arxiv.org/pdf/2001.10048v1.pdf
PWC	https://paperswithcode.com/paper/an-ontology-aware-framework-for-audio-event
Repo
Framework

Second Order Optimization Made Practical


Title	Second Order Optimization Made Practical
Authors	Rohan Anil, Vineet Gupta, Tomer Koren, Kevin Regan, Yoram Singer
Abstract	Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Second-order optimization methods that involve second-order derivatives and/or second-order statistics of the data have become far less prevalent despite strong theoretical properties, due to their prohibitive computation, memory and communication costs. In an attempt to bridge this gap between theoretical and practical optimization, we present a proof-of-concept distributed system implementation of a second-order preconditioned method (specifically, a variant of full-matrix Adagrad), that along with a few yet critical algorithmic and numerical improvements, provides significant practical gains in convergence on state-of-the-art deep models and gives rise to actual wall-time improvements in practice compared to conventional first-order methods. Our design effectively utilizes the prevalent heterogeneous hardware architecture for training deep models which consists of a multicore CPU coupled with multiple accelerator units. We demonstrate superior performance on very large learning problems in machine translation where our distributed implementation runs considerably faster than existing gradient-based methods.
Tasks	Machine Translation
Published	2020-02-20
URL	https://arxiv.org/abs/2002.09018v1
PDF	https://arxiv.org/pdf/2002.09018v1.pdf
PWC	https://paperswithcode.com/paper/second-order-optimization-made-practical
Repo
Framework

Technical Background for “A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis”


Title	Technical Background for “A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis”
Authors	Xiaotong Jiang, Amanda E. Nelson, Rebecca J. Cleveland, Daniel P. Beavers, Todd A. Schwartz, Liubov Arbeeva, Carolina Alvarez, Leigh F. Callahan, Stephen Messier, Richard Loeser, Michael R. Kosorok
Abstract	We provide additional statistical background for the methodology developed in the clinical analysis of knee osteoarthritis in “A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis” (Jiang et al. 2020). Jiang et al. 2020 proposed a pipeline to learn optimal treatment rules with precision medicine models and compared them with zero-order models with a Z-test. The model performance was based on value functions, a scalar that predicts the future reward of each decision rule. The jackknife (i.e., leave-one-out cross validation) method was applied to estimate the value function and its variance of several outcomes in IDEA. IDEA is a randomized clinical trial studying three interventions (exercise (E), dietary weight loss (D), and D+E) on overweight and obese participants with knee osteoarthritis. In this report, we expand the discussion and justification with additional statistical background. We elaborate more on the background of precision medicine, the derivation of the jackknife estimator of value function and its estimated variance, the consistency property of jackknife estimator, as well as additional simulation results that reflect more of the performance of jackknife estimators. We recommend reading Jiang et al. 2020 for clinical application and interpretation of the optimal ITR of knee osteoarthritis as well as the overall understanding of the pipeline and recommend using this article to understand the underlying statistical derivation and methodology.
Tasks
Published	2020-01-27
URL	https://arxiv.org/abs/2001.09930v3
PDF	https://arxiv.org/pdf/2001.09930v3.pdf
PWC	https://paperswithcode.com/paper/a-precision-medicine-approach-to-develop-and
Repo
Framework

The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks


Title	The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks
Authors	Jakub Swiatkowski, Kevin Roth, Bastiaan S. Veeling, Linh Tran, Joshua V. Dillon, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, Sebastian Nowozin
Abstract	Variational Bayesian Inference is a popular methodology for approximating posterior distributions over Bayesian neural network weights. Recent work developing this class of methods has explored ever richer parameterizations of the approximate posterior in the hope of improving performance. In contrast, here we share a curious experimental finding that suggests instead restricting the variational distribution to a more compact parameterization. For a variety of deep Bayesian neural networks trained using Gaussian mean-field variational inference, we find that the posterior standard deviations consistently exhibit strong low-rank structure after convergence. This means that by decomposing these variational parameters into a low-rank factorization, we can make our variational approximation more compact without decreasing the models’ performance. Furthermore, we find that such factorized parameterizations improve the signal-to-noise ratio of stochastic gradient estimates of the variational lower bound, resulting in faster convergence.
Tasks	Bayesian Inference
Published	2020-02-07
URL	https://arxiv.org/abs/2002.02655v1
PDF	https://arxiv.org/pdf/2002.02655v1.pdf
PWC	https://paperswithcode.com/paper/the-k-tied-normal-distribution-a-compact
Repo
Framework

Detection and Recovery of Adversarial Attacks with Injected Attractors


Title	Detection and Recovery of Adversarial Attacks with Injected Attractors
Authors	Jiyi Zhang, Ee-Chien Chang, Hwee Kuan Lee
Abstract	Many machine learning adversarial attacks find adversarial samples of a victim model ${\mathcal M}$ by following the gradient of some functions, either explicitly or implicitly. To detect and recover from such attacks, we take the proactive approach that modifies those functions with the goal of misleading the attacks to some local minimals, or to some designated regions that can be easily picked up by a forensic analyzer. To achieve the goal, we propose adding a large number of artifacts, which we called $attractors$, onto the otherwise smooth function. An attractor is a point in the input space, which has a neighborhood of samples with gradients pointing toward it. We observe that decoders of watermarking schemes exhibit properties of attractors, and give a generic method that injects attractors from a watermark decoder into the victim model ${\mathcal M}$. This principled approach allows us to leverage on known watermarking schemes for scalability and robustness. Experimental studies show that our method has competitive performance. For instance, for un-targeted attacks on CIFAR-10 dataset, we can reduce the overall attack success rate of DeepFool to 1.9%, whereas known defence LID, FS and MagNet can reduce the rate to 90.8%, 98.5% and 78.5% respectively.
Tasks
Published	2020-03-05
URL	https://arxiv.org/abs/2003.02732v1
PDF	https://arxiv.org/pdf/2003.02732v1.pdf
PWC	https://paperswithcode.com/paper/detection-and-recovery-of-adversarial-attacks
Repo
Framework

Rectified Meta-Learning from Noisy Labels for Robust Image-based Plant Disease Diagnosis


Title	Rectified Meta-Learning from Noisy Labels for Robust Image-based Plant Disease Diagnosis
Authors	Ruifeng Shi, Deming Zhai, Xianming Liu, Junjun Jiang, Wen Gao
Abstract	Plant diseases serve as one of main threats to food security and crop production. It is thus valuable to exploit recent advances of artificial intelligence to assist plant disease diagnosis. One popular approach is to transform this problem as a leaf image classification task, which can be then addressed by the powerful convolutional neural networks (CNNs). However, the performance of CNN-based classification approach depends on a large amount of high-quality manually labeled training data, which are inevitably introduced noise on labels in practice, leading to model overfitting and performance degradation. To overcome this problem, we propose a novel framework that incorporates rectified meta-learning module into common CNN paradigm to train a noise-robust deep network without using extra supervision information. The proposed method enjoys the following merits: i) A rectified meta-learning is designed to pay more attention to unbiased samples, leading to accelerated convergence and improved classification accuracy. ii) Our method is free on assumption of label noise distribution, which works well on various kinds of noise. iii) Our method serves as a plug-and-play module, which can be embedded into any deep models optimized by gradient descent based method. Extensive experiments are conducted to demonstrate the superior performance of our algorithm over the state-of-the-arts.
Tasks	Image Classification, Meta-Learning
Published	2020-03-17
URL	https://arxiv.org/abs/2003.07603v2
PDF	https://arxiv.org/pdf/2003.07603v2.pdf
PWC	https://paperswithcode.com/paper/rectified-meta-learning-from-noisy-labels-for
Repo
Framework