April 3, 2020

3337 words 16 mins read

Paper Group AWR 37

A Neuro-AI Interface for Evaluating Generative Adversarial Networks. batchboost: regularization for stabilizing training with resistance to underfitting & overfitting. Learning Contextualized Document Representations for Healthcare Answer Retrieval. Improving Neural Named Entity Recognition with Gazetteers. TyDi QA: A Benchmark for Information-Seek …

A Neuro-AI Interface for Evaluating Generative Adversarial Networks


Title	A Neuro-AI Interface for Evaluating Generative Adversarial Networks
Authors	Zhengwei Wang, Qi She, Alan F. Smeaton, Tomas E. Ward, Graham Healy
Abstract	Generative adversarial networks (GANs) are increasingly attracting attention in the computer vision, natural language processing, speech synthesis and similar domains. However, evaluating the performance of GANs is still an open and challenging problem. Existing evaluation metrics primarily measure the dissimilarity between real and generated images using automated statistical methods. They often require large sample sizes for evaluation and do not directly reflect human perception of image quality. In this work, we introduce an evaluation metric called Neuroscore, for evaluating the performance of GANs, that more directly reflects psychoperceptual image quality through the utilization of brain signals. Our results show that Neuroscore has superior performance to the current evaluation metrics in that: (1) It is more consistent with human judgment; (2) The evaluation process needs much smaller numbers of samples; and (3) It is able to rank the quality of images on a per GAN basis. A convolutional neural network (CNN) based neuro-AI interface is proposed to predict Neuroscore from GAN-generated images directly without the need for neural responses. Importantly, we show that including neural responses during the training phase of the network can significantly improve the prediction capability of the proposed model. Codes and data can be referred at this link: https://github.com/villawang/Neuro-AI-Interface.
Tasks	Speech Synthesis
Published	2020-03-05
URL	https://arxiv.org/abs/2003.03193v1
PDF	https://arxiv.org/pdf/2003.03193v1.pdf
PWC	https://paperswithcode.com/paper/a-neuro-ai-interface-for-evaluating
Repo	https://github.com/villawang/Neuro-AI-Interface
Framework	tf

batchboost: regularization for stabilizing training with resistance to underfitting & overfitting


Title	batchboost: regularization for stabilizing training with resistance to underfitting & overfitting
Authors	Maciej A. Czyzewski
Abstract	Overfitting & underfitting and stable training are an important challenges in machine learning. Current approaches for these issues are mixup, SamplePairing and BC learning. In our work, we state the hypothesis that mixing many images together can be more effective than just two. Batchboost pipeline has three stages: (a) pairing: method of selecting two samples. (b) mixing: how to create a new one from two samples. (c) feeding: combining mixed samples with new ones from dataset into batch (with ratio $\gamma$). Note that sample that appears in our batch propagates with subsequent iterations with less and less importance until the end of training. Pairing stage calculates the error per sample, sorts the samples and pairs with strategy: hardest with easiest one, than mixing stage merges two samples using mixup, $x_1 + (1-\lambda)x_2$. Finally, feeding stage combines new samples with mixed by ratio 1:1. Batchboost has 0.5-3% better accuracy than the current state-of-the-art mixup regularization on CIFAR-10 & Fashion-MNIST. Our method is slightly better than SamplePairing technique on small datasets (up to 5%). Batchboost provides stable training on not tuned parameters (like weight decay), thus its a good method to test performance of different architectures. Source code is at: https://github.com/maciejczyzewski/batchboost
Tasks	Image Classification, Semi-Supervised Image Classification
Published	2020-01-21
URL	https://arxiv.org/abs/2001.07627v1
PDF	https://arxiv.org/pdf/2001.07627v1.pdf
PWC	https://paperswithcode.com/paper/batchboost-regularization-for-stabilizing
Repo	https://github.com/maciejczyzewski/batchboost
Framework	pytorch

Learning Contextualized Document Representations for Healthcare Answer Retrieval


Title	Learning Contextualized Document Representations for Healthcare Answer Retrieval
Authors	Sebastian Arnold, Betty van Aken, Paul Grundmann, Felix A. Gers, Alexander Löser
Abstract	We present Contextual Discourse Vectors (CDV), a distributed document representation for efficient answer retrieval from long healthcare documents. Our approach is based on structured query tuples of entities and aspects from free text and medical taxonomies. Our model leverages a dual encoder architecture with hierarchical LSTM layers and multi-task training to encode the position of clinical entities and aspects alongside the document discourse. We use our continuous representations to resolve queries with short latency using approximate nearest neighbor search on sentence level. We apply the CDV model for retrieving coherent answer passages from nine English public health resources from the Web, addressing both patients and medical professionals. Because there is no end-to-end training data available for all application scenarios, we train our model with self-supervised data from Wikipedia. We show that our generalized model significantly outperforms several state-of-the-art baselines for healthcare passage ranking and is able to adapt to heterogeneous domains without additional fine-tuning.
Tasks
Published	2020-02-03
URL	https://arxiv.org/abs/2002.00835v1
PDF	https://arxiv.org/pdf/2002.00835v1.pdf
PWC	https://paperswithcode.com/paper/learning-contextualized-document
Repo	https://github.com/sebastianarnold/cdv
Framework	none

Improving Neural Named Entity Recognition with Gazetteers


Title	Improving Neural Named Entity Recognition with Gazetteers
Authors	Chan Hee Song, Dawn Lawrie, Tim Finin, James Mayfield
Abstract	The goal of this work is to improve the performance of a neural named entity recognition system by adding input features that indicate a word is part of a name included in a gazetteer. This article describes how to generate gazetteers from the Wikidata knowledge graph as well as how to integrate the information into a neural NER system. Experiments reveal that the approach yields performance gains in two distinct languages: a high-resource, word-based language, English and a high-resource, character-based language, Chinese. Experiments were also performed in a low-resource language, Russian on a newly annotated Russian NER corpus from Reddit tagged with four core types and twelve extended types. This article reports a baseline score. It is a longer version of a paper in the 33rd FLAIRS conference (Song et al. 2020).
Tasks	Named Entity Recognition
Published	2020-03-06
URL	https://arxiv.org/abs/2003.03072v1
PDF	https://arxiv.org/pdf/2003.03072v1.pdf
PWC	https://paperswithcode.com/paper/improving-neural-named-entity-recognition
Repo	https://github.com/hltcoe/gazetteer-collection
Framework	none

TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages


Title	TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages
Authors	Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, Jennimaria Palomaki
Abstract	Confidently making progress on multilingual modeling requires challenging, trustworthy evaluations. We present TyDi QA—a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs. The languages of TyDi QA are diverse with regard to their typology—the set of linguistic features each language expresses—such that we expect models performing well on this set to generalize across a large number of the world’s languages. We present a quantitative analysis of the data quality and example-level qualitative linguistic analyses of observed language phenomena that would not be found in English-only corpora. To provide a realistic information-seeking task and avoid priming effects, questions are written by people who want to know the answer, but don’t know the answer yet, and the data is collected directly in each language without the use of translation.
Tasks	Question Answering
Published	2020-03-10
URL	https://arxiv.org/abs/2003.05002v1
PDF	https://arxiv.org/pdf/2003.05002v1.pdf
PWC	https://paperswithcode.com/paper/tydi-qa-a-benchmark-for-information-seeking
Repo	https://github.com/google-research-datasets/tydiqa
Framework	tf

Weakly-Supervised Salient Object Detection via Scribble Annotations


Title	Weakly-Supervised Salient Object Detection via Scribble Annotations
Authors	Jing Zhang, Xin Yu, Aixuan Li, Peipei Song, Bowen Liu, Yuchao Dai
Abstract	Compared with laborious pixel-wise dense labeling, it is much easier to label data by scribbles, which only costs 1$\sim$2 seconds to label one image. However, using scribble labels to learn salient object detection has not been explored. In this paper, we propose a weakly-supervised salient object detection model to learn saliency from such annotations. In doing so, we first relabel an existing large-scale salient object detection dataset with scribbles, namely S-DUTS dataset. Since object structure and detail information is not identified by scribbles, directly training with scribble labels will lead to saliency maps of poor boundary localization. To mitigate this problem, we propose an auxiliary edge detection task to localize object edges explicitly, and a gated structure-aware loss to place constraints on the scope of structure to be recovered. Moreover, we design a scribble boosting scheme to iteratively consolidate our scribble annotations, which are then employed as supervision to learn high-quality saliency maps. As existing saliency evaluation metrics neglect to measure structure alignment of the predictions, the saliency map ranking metric may not comply with human perception. We present a new metric, termed saliency structure measure, to measure the structure alignment of the predicted saliency maps, which is more consistent with human perception. Extensive experiments on six benchmark datasets demonstrate that our method not only outperforms existing weakly-supervised/unsupervised methods, but also is on par with several fully-supervised state-of-the-art models. Our code and data is publicly available at https://github.com/JingZhang617/Scribble_Saliency.
Tasks	Edge Detection, Object Detection, Salient Object Detection
Published	2020-03-17
URL	https://arxiv.org/abs/2003.07685v1
PDF	https://arxiv.org/pdf/2003.07685v1.pdf
PWC	https://paperswithcode.com/paper/weakly-supervised-salient-object-detection-1
Repo	https://github.com/JingZhang617/Scribble_Saliency
Framework	none

jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models


Title	jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
Authors	Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, Samuel R. Bowman
Abstract	We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. jiant enables modular and configuration-driven experimentation with state-of-the-art models and implements a broad set of tasks for probing, transfer learning, and multitask training experiments. jiant implements over 50 NLU tasks, including all GLUE and SuperGLUE benchmark tasks. We demonstrate that jiant reproduces published performance on a variety of tasks and models, including BERT and RoBERTa. jiant is available at https://jiant.info.
Tasks	Transfer Learning
Published	2020-03-04
URL	https://arxiv.org/abs/2003.02249v1
PDF	https://arxiv.org/pdf/2003.02249v1.pdf
PWC	https://paperswithcode.com/paper/jiant-a-software-toolkit-for-research-on
Repo	https://github.com/nyu-mll/jiant
Framework	pytorch

The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification


Title	The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification
Authors	Dongliang Chang, Yifeng Ding, Jiyang Xie, Ayan Kumar Bhunia, Xiaoxu Li, Zhanyu Ma, Ming Wu, Jun Guo, Yi-Zhe Song
Abstract	Key for solving fine-grained image categorization is finding discriminate and local regions that correspond to subtle visual traits. Great strides have been made, with complex networks designed specifically to learn part-level discriminate feature representations. In this paper, we show it is possible to cultivate subtle details without the need for overly complicated network designs or training mechanisms – a single loss is all it takes. The main trick lies with how we delve into individual feature channels early on, as opposed to the convention of starting from a consolidated feature map. The proposed loss function, termed as mutual-channel loss (MC-Loss), consists of two channel-specific components: a discriminality component and a diversity component. The discriminality component forces all feature channels belonging to the same class to be discriminative, through a novel channel-wise attention mechanism. The diversity component additionally constraints channels so that they become mutually exclusive on spatial-wise. The end result is therefore a set of feature channels that each reflects different locally discriminative regions for a specific class. The MC-Loss can be trained end-to-end, without the need for any bounding-box/part annotations, and yields highly discriminative regions during inference. Experimental results show our MC-Loss when implemented on top of common base networks can achieve state-of-the-art performance on all four fine-grained categorization datasets (CUB-Birds, FGVC-Aircraft, Flowers-102, and Stanford-Cars). Ablative studies further demonstrate the superiority of MC-Loss when compared with other recently proposed general-purpose losses for visual classification, on two different base networks. Code available at https://github.com/dongliangchang/Mutual-Channel-Loss
Tasks	Fine-Grained Image Classification, Image Categorization, Image Classification
Published	2020-02-11
URL	https://arxiv.org/abs/2002.04264v1
PDF	https://arxiv.org/pdf/2002.04264v1.pdf
PWC	https://paperswithcode.com/paper/the-devil-is-in-the-channels-mutual-channel
Repo	https://github.com/dongliangchang/Mutual-Channel-Loss
Framework	pytorch

Geom-GCN: Geometric Graph Convolutional Networks


Title	Geom-GCN: Geometric Graph Convolutional Networks
Authors	Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, Bo Yang
Abstract	Message-passing neural networks (MPNNs) have been successfully applied to representation learning on graphs in a variety of real-world applications. However, two fundamental weaknesses of MPNNs’ aggregators limit their ability to represent graph-structured data: losing the structural information of nodes in neighborhoods and lacking the ability to capture long-range dependencies in disassortative graphs. Few studies have noticed the weaknesses from different perspectives. From the observations on classical neural network and network geometry, we propose a novel geometric aggregation scheme for graph neural networks to overcome the two weaknesses. The behind basic idea is the aggregation on a graph can benefit from a continuous space underlying the graph. The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation. We also present an implementation of the scheme in graph convolutional networks, termed Geom-GCN (Geometric Graph Convolutional Networks), to perform transductive learning on graphs. Experimental results show the proposed Geom-GCN achieved state-of-the-art performance on a wide range of open datasets of graphs. Code is available at https://github.com/graphdml-uiuc-jlu/geom-gcn.
Tasks	Representation Learning
Published	2020-02-13
URL	https://arxiv.org/abs/2002.05287v2
PDF	https://arxiv.org/pdf/2002.05287v2.pdf
PWC	https://paperswithcode.com/paper/geom-gcn-geometric-graph-convolutional-1
Repo	https://github.com/graphdml-uiuc-jlu/geom-gcn
Framework	pytorch

Survival Cluster Analysis


Title	Survival Cluster Analysis
Authors	Paidamoyo Chapfuwa, Chunyuan Li, Nikhil Mehta, Lawrence Carin, Ricardo Henao
Abstract	Conventional survival analysis approaches estimate risk scores or individualized time-to-event distributions conditioned on covariates. In practice, there is often great population-level phenotypic heterogeneity, resulting from (unknown) subpopulations with diverse risk profiles or survival distributions. As a result, there is an unmet need in survival analysis for identifying subpopulations with distinct risk profiles, while jointly accounting for accurate individualized time-to-event predictions. An approach that addresses this need is likely to improve characterization of individual outcomes by leveraging regularities in subpopulations, thus accounting for population-level heterogeneity. In this paper, we propose a Bayesian nonparametrics approach that represents observations (subjects) in a clustered latent space, and encourages accurate time-to-event predictions and clusters (subpopulations) with distinct risk profiles. Experiments on real-world datasets show consistent improvements in predictive performance and interpretability relative to existing state-of-the-art survival analysis models.
Tasks	Survival Analysis
Published	2020-02-29
URL	https://arxiv.org/abs/2003.00355v1
PDF	https://arxiv.org/pdf/2003.00355v1.pdf
PWC	https://paperswithcode.com/paper/survival-cluster-analysis
Repo	https://github.com/paidamoyo/survival_cluster_analysis
Framework	tf

Tracing patients’ PLOD with mobile phones: Mitigation of epidemic risks through patients’ locational open data


Title	Tracing patients’ PLOD with mobile phones: Mitigation of epidemic risks through patients’ locational open data
Authors	Ikki Ohmukai, Yasunori Yamamoto, Maori Ito, Takashi Okumura
Abstract	In the cases when public health authorities confirm a patient with highly contagious disease, they release the summaries about patient locations and travel information. However, due to privacy concerns, these releases do not include the detailed data and typically comprise the information only about commercial facilities and public transportation used by the patients. We addressed this problem and proposed to release the patient location data as open data represented in a structured form of the information described in press releases. Therefore, residents would be able to use these data for automated estimation of the potential risks of contacts combined with the location information stored in their mobile phones. This paper proposes the design of the open data based on Resource Description Framework (RDF), and performs a preliminary evaluation of the first draft of the specification followed by a discussion on possible future directions.
Tasks
Published	2020-03-13
URL	https://arxiv.org/abs/2003.06199v1
PDF	https://arxiv.org/pdf/2003.06199v1.pdf
PWC	https://paperswithcode.com/paper/tracing-patients-plod-with-mobile-phones
Repo	https://github.com/tanupoo/penguin
Framework	none

3D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation


Title	3D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation
Authors	Iñigo Alonso, Luis Riazuelo, Luis Montesano, Ana C. Murillo
Abstract	LIDAR semantic segmentation, which assigns a semantic label to each 3D point measured by the LIDAR, is becoming an essential task for many robotic applications such as autonomous driving. Fast and efficient semantic segmentation methods are needed to match the strong computational and temporal restrictions of many of these real-world applications. This work presents 3D-MiniNet, a novel approach for LIDAR semantic segmentation that combines 3D and 2D learning layers. It first learns a 2D representation from the raw points through a novel projection which extracts local and global information from the 3D data. This representation is fed to an efficient 2D Fully Convolutional Neural Network (FCNN) that produces a 2D semantic segmentation. These 2D semantic labels are re-projected back to the 3D space and enhanced through a post-processing module. The main novelty in our strategy relies on the projection learning module. Our detailed ablation study shows how each component contributes to the final performance of 3D-MiniNet. We validate our approach on well known public benchmarks (SemanticKITTI and KITTI), where 3D-MiniNet gets state-of-the-art results while being faster and more parameter-efficient than previous methods.
Tasks	3D Semantic Segmentation, Autonomous Driving, Autonomous Vehicles, Real-Time 3D Semantic Segmentation, Real-Time Semantic Segmentation, Semantic Segmentation
Published	2020-02-25
URL	https://arxiv.org/abs/2002.10893v2
PDF	https://arxiv.org/pdf/2002.10893v2.pdf
PWC	https://paperswithcode.com/paper/3d-mininet-learning-a-2d-representation-from
Repo	https://github.com/Shathe/3D-MiniNet
Framework	tf

Bidimensional linked matrix factorization for pan-omics pan-cancer analysis


Title	Bidimensional linked matrix factorization for pan-omics pan-cancer analysis
Authors	Eric F. Lock, Jun Young Park, Katherine A. Hoadley
Abstract	Several modern applications require the integration of multiple large data matrices that have shared rows and/or columns. For example, cancer studies that integrate multiple omics platforms across multiple types of cancer, pan-omics pan-cancer analysis, have extended our knowledge of molecular heterogenity beyond what was observed in single tumor and single platform studies. However, these studies have been limited by available statistical methodology. We propose a flexible approach to the simultaneous factorization and decomposition of variation across such bidimensionally linked matrices, BIDIFAC+. This decomposes variation into a series of low-rank components that may be shared across any number of row sets (e.g., omics platforms) or column sets (e.g., cancer types). This builds on a growing literature for the factorization and decomposition of linked matrices, which has primarily focused on multiple matrices that are linked in one dimension (rows or columns) only. Our objective function extends nuclear norm penalization, is motivated by random matrix theory, gives an identifiable decomposition under relatively mild conditions, and can be shown to give the mode of a Bayesian posterior distribution. We apply BIDIFAC+ to pan-omics pan-cancer data from TCGA, identifying shared and specific modes of variability across 4 different omics platforms and 29 different cancer types.
Tasks
Published	2020-02-07
URL	https://arxiv.org/abs/2002.02601v1
PDF	https://arxiv.org/pdf/2002.02601v1.pdf
PWC	https://paperswithcode.com/paper/bidimensional-linked-matrix-factorization-for
Repo	https://github.com/lockEF/bidifac
Framework	none

A Survey on Knowledge Graphs: Representation, Acquisition and Applications


Title	A Survey on Knowledge Graphs: Representation, Acquisition and Applications
Authors	Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, Philip S. Yu
Abstract	Human knowledge provides a formal understanding of the world. Knowledge graphs that represent structural relations between entities have become an increasingly popular research direction towards cognition and human-level intelligence. In this survey, we provide a comprehensive review on knowledge graph covering overall research topics about 1) knowledge graph representation learning, 2) knowledge acquisition and completion, 3) temporal knowledge graph, and 4) knowledge-aware applications, and summarize recent breakthroughs and perspective directions to facilitate future research. We propose a full-view categorization and new taxonomies on these topics. Knowledge graph embedding is organized from four aspects of representation space, scoring function, encoding models and auxiliary information. For knowledge acquisition, especially knowledge graph completion, embedding methods, path inference and logical rule reasoning are reviewed. We further explore several emerging topics including meta relational learning, commonsense reasoning, and temporal knowledge graphs. To facilitate future research on knowledge graphs, we also provide a curated collection of datasets and open-source libraries on different tasks. In the end, we have a thorough outlook on several promising research directions.
Tasks	Graph Embedding, Graph Representation Learning, Knowledge Graph Completion, Knowledge Graph Embedding, Knowledge Graphs, Relational Reasoning, Representation Learning
Published	2020-02-02
URL	https://arxiv.org/abs/2002.00388v1
PDF	https://arxiv.org/pdf/2002.00388v1.pdf
PWC	https://paperswithcode.com/paper/a-survey-on-knowledge-graphs-representation
Repo	https://github.com/pizzaonline/Knowledge-Graph
Framework	none

Lagrangian Neural Networks


Title	Lagrangian Neural Networks
Authors	Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, Shirley Ho
Abstract	Accurate models of the world are built upon notions of its underlying symmetries. In physics, these symmetries correspond to conservation laws, such as for energy and momentum. Yet even though neural network models see increasing use in the physical sciences, they struggle to learn these symmetries. In this paper, we propose Lagrangian Neural Networks (LNNs), which can parameterize arbitrary Lagrangians using neural networks. In contrast to models that learn Hamiltonians, LNNs do not require canonical coordinates, and thus perform well in situations where canonical momenta are unknown or difficult to compute. Unlike previous approaches, our method does not restrict the functional form of learned energies and will produce energy-conserving models for a variety of tasks. We test our approach on a double pendulum and a relativistic particle, demonstrating energy conservation where a baseline approach incurs dissipation and modeling relativity without canonical coordinates where a Hamiltonian approach fails. Finally, we show how this model can be applied to graphs and continuous systems using a Lagrangian Graph Network, and demonstrate it on the 1D wave equation.
Tasks
Published	2020-03-10
URL	https://arxiv.org/abs/2003.04630v1
PDF	https://arxiv.org/pdf/2003.04630v1.pdf
PWC	https://paperswithcode.com/paper/lagrangian-neural-networks
Repo	https://github.com/MilesCranmer/lagrangian_nns
Framework	jax