January 28, 2020

3228 words 16 mins read

Paper Group ANR 870

Population-aware Hierarchical Bayesian Domain Adaptation via Multiple-component Invariant Learning. What can computational models learn from human selective attention? A review from an audiovisual crossmodal perspective. Unwanted Advances in Higher Education: Uncovering Sexual Harassment Experiences in Academia with Text Mining. Training Deep Neura …

Population-aware Hierarchical Bayesian Domain Adaptation via Multiple-component Invariant Learning


Title	Population-aware Hierarchical Bayesian Domain Adaptation via Multiple-component Invariant Learning
Authors	Vishwali Mhasawade, Nabeel Abdur Rehman, Rumi Chunara
Abstract	While machine learning is rapidly being developed and deployed in health settings such as influenza prediction, there are critical challenges in using data from one environment in another due to variability in features; even within disease labels there can be differences (e.g. “fever” may mean something different reported in a doctor’s office versus in an online app). Moreover, models are often built on passive, observational data which contain different distributions of population subgroups (e.g. men or women). Thus, there are two forms of instability between environments in this observational transport problem. We first harness knowledge from health to conceptualize the underlying causal structure of this problem in a health outcome prediction task. Based on sources of stability in the model, we posit that for human-sourced data and health prediction tasks we can combine environment and population information in a novel population-aware hierarchical Bayesian domain adaptation framework that harnesses multiple invariant components through population attributes when needed. We study the conditions under which invariant learning fails, leading to reliance on the environment-specific attributes. Experimental results for an influenza prediction task on four datasets gathered from different contexts show the model can improve prediction in the case of largely unlabelled target data from a new environment and different constituent population, by harnessing both environment and population invariant information. This work represents a novel, principled way to address a critical challenge by blending domain (health) knowledge and algorithmic innovation. The proposed approach will have a significant impact in many social settings wherein who and where the data comes from matters.
Tasks	Domain Adaptation
Published	2019-08-24
URL	https://arxiv.org/abs/1908.09222v5
PDF	https://arxiv.org/pdf/1908.09222v5.pdf
PWC	https://paperswithcode.com/paper/population-aware-hierarchical-bayesian-domain-1
Repo
Framework

What can computational models learn from human selective attention? A review from an audiovisual crossmodal perspective


Title	What can computational models learn from human selective attention? A review from an audiovisual crossmodal perspective
Authors	Di Fu, Cornelius Weber, Guochun Yang, Matthias Kerzel, Weizhi Nan, Pablo Barros, Haiyan Wu, Xun Liu, Stefan Wermter
Abstract	Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans’ behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for computational intelligent agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdisciplinary review and provide insights about how to use psychological findings and theories in artificial intelligence from different perspectives.
Tasks
Published	2019-09-05
URL	https://arxiv.org/abs/1909.05654v1
PDF	https://arxiv.org/pdf/1909.05654v1.pdf
PWC	https://paperswithcode.com/paper/what-can-computational-models-learn-from
Repo
Framework

Unwanted Advances in Higher Education: Uncovering Sexual Harassment Experiences in Academia with Text Mining


Title	Unwanted Advances in Higher Education: Uncovering Sexual Harassment Experiences in Academia with Text Mining
Authors	Amir Karami, Cynthia Nicole White, Kayla Ford, Suzanne Swan, Melek Yildiz Spinel
Abstract	Sexual harassment in academia is often a hidden problem because victims are usually reluctant to report their experiences. Recently, a web survey was developed to provide an opportunity to share thousands of sexual harassment experiences in academia. Using an efficient approach, this study collected and investigated more than 2,000 sexual harassment experiences to better understand these unwanted advances in higher education. This paper utilized text mining to disclose hidden topics and explore their weight across three variables: harasser gender, institution type, and victim’s field of study. We mapped the topics on five themes drawn from the sexual harassment literature and found that more than 50% of the topics were assigned to the unwanted sexual attention theme. Fourteen percent of the topics were in the gender harassment theme, in which insulting, sexist, or degrading comments or behavior was directed towards women. Five percent of the topics involved sexual coercion (a benefit is offered in exchange for sexual favors), 5% involved sex discrimination, and 7% of the topics discussed retaliation against the victim for reporting the harassment, or for simply not complying with the harasser. Findings highlight the power differential between faculty and students, and the toll on students when professors abuse their power. While some topics did differ based on type of institution, there were no differences between the topics based on gender of harasser or field of study. This research can be beneficial to researchers in further investigation of this paper’s dataset, and to policymakers in improving existing policies to create a safe and supportive environment in academia.
Tasks
Published	2019-12-11
URL	https://arxiv.org/abs/2001.11552v1
PDF	https://arxiv.org/pdf/2001.11552v1.pdf
PWC	https://paperswithcode.com/paper/unwanted-advances-in-higher-education
Repo
Framework

Training Deep Neural Networks Using Posit Number System


Title	Training Deep Neural Networks Using Posit Number System
Authors	Jinming Lu, Siyuan Lu, Zhisheng Wang, Chao Fang, Jun Lin, Zhongfeng Wang, Li Du
Abstract	With the increasing size of Deep Neural Network (DNN) models, the high memory space requirements and computational complexity have become an obstacle for efficient DNN implementations. To ease this problem, using reduced-precision representations for DNN training and inference has attracted many interests from researchers. This paper first proposes a methodology for training DNNs with the posit arithmetic, a type- 3 universal number (Unum) format that is similar to the floating point(FP) but has reduced precision. A warm-up training strategy and layer-wise scaling factors are adopted to stabilize training and fit the dynamic range of DNN parameters. With the proposed training methodology, we demonstrate the first successful training of DNN models on ImageNet image classification task in 16 bits posit with no accuracy loss. Then, an efficient hardware architecture for the posit multiply-and-accumulate operation is also proposed, which can achieve significant improvement in energy efficiency than traditional floating-point implementations. The proposed design is helpful for future low-power DNN training accelerators.
Tasks	Image Classification
Published	2019-09-06
URL	https://arxiv.org/abs/1909.03831v1
PDF	https://arxiv.org/pdf/1909.03831v1.pdf
PWC	https://paperswithcode.com/paper/training-deep-neural-networks-using-posit
Repo
Framework

Integration of Convolutional Neural Networks for Pulmonary Nodule Malignancy Assessment in a Lung Cancer Classification Pipeline


Title	Integration of Convolutional Neural Networks for Pulmonary Nodule Malignancy Assessment in a Lung Cancer Classification Pipeline
Authors	Ilaria Bonavita, Xavier Rafael-Palou, Mario Ceresa, Gemma Piella, Vicent Ribas, Miguel A. González Ballester
Abstract	The early identification of malignant pulmonary nodules is critical for better lung cancer prognosis and less invasive chemo or radio therapies. Nodule malignancy assessment done by radiologists is extremely useful for planning a preventive intervention but is, unfortunately, a complex, time-consuming and error-prone task. This explains the lack of large datasets containing radiologists malignancy characterization of nodules. In this article, we propose to assess nodule malignancy through 3D convolutional neural networks and to integrate it in an automated end-to-end existing pipeline of lung cancer detection. For training and testing purposes we used independent subsets of the LIDC dataset. Adding the probabilities of nodules malignity in a baseline lung cancer pipeline improved its F1-weighted score by 14.7%, whereas integrating the malignancy model itself using transfer learning outperformed the baseline prediction by 11.8% of F1-weighted score. Despite the limited size of the lung cancer datasets, integrating predictive models of nodule malignancy improves prediction of lung cancer.
Tasks	Transfer Learning
Published	2019-12-18
URL	https://arxiv.org/abs/1912.08679v1
PDF	https://arxiv.org/pdf/1912.08679v1.pdf
PWC	https://paperswithcode.com/paper/integration-of-convolutional-neural-networks
Repo
Framework

Deep AutoEncoder-based Lossy Geometry Compression for Point Clouds


Title	Deep AutoEncoder-based Lossy Geometry Compression for Point Clouds
Authors	Wei Yan, Yiting shao, Shan Liu, Thomas H Li, Zhu Li, Ge Li
Abstract	Point cloud is a fundamental 3D representation which is widely used in real world applications such as autonomous driving. As a newly-developed media format which is characterized by complexity and irregularity, point cloud creates a need for compression algorithms which are more flexible than existing codecs. Recently, autoencoders(AEs) have shown their effectiveness in many visual analysis tasks as well as image compression, which inspires us to employ it in point cloud compression. In this paper, we propose a general autoencoder-based architecture for lossy geometry point cloud compression. To the best of our knowledge, it is the first autoencoder-based geometry compression codec that directly takes point clouds as input rather than voxel grids or collections of images. Compared with handcrafted codecs, this approach adapts much more quickly to previously unseen media contents and media formats, meanwhile achieving competitive performance. Our architecture consists of a pointnet-based encoder, a uniform quantizer, an entropy estimation block and a nonlinear synthesis transformation module. In lossy geometry compression of point cloud, results show that the proposed method outperforms the test model for categories 1 and 3 (TMC13) published by MPEG-3DG group on the 125th meeting, and on average a 73.15% BD-rate gain is achieved.
Tasks	Autonomous Driving, Image Compression
Published	2019-04-18
URL	http://arxiv.org/abs/1905.03691v1
PDF	http://arxiv.org/pdf/1905.03691v1.pdf
PWC	https://paperswithcode.com/paper/190503691
Repo
Framework

BAS: An Answer Selection Method Using BERT Language Model


Title	BAS: An Answer Selection Method Using BERT Language Model
Authors	Jamshid Mozafari, Afsaneh Fatemi, Mohammad Ali Nematbakhsh
Abstract	In recent years, Question Answering systems have become more popular and widely used by users. Despite the increasing popularity of these systems, the their performance is not even sufficient for textual data and requires further research. These systems consist of several parts that one of them is the Answer Selection component. This component detects the most relevant answer from a list of candidate answers. The methods presented in previous researches have attempted to provide an independent model to undertake the answer-selection task. An independent model cannot comprehend the syntactic and semantic features of questions and answers with a small training dataset. To fill this gap, language models can be employed in implementing the answer selection part. This action enables the model to have a better understanding of the language in order to understand questions and answers better than previous works. In this research, we will present the “BAS” (BERT Answer Selection) that uses the BERT language model to comprehend language. The empirical results of applying the model on the TrecQA Raw, TrecQA Clean, and WikiQA datasets demonstrate that using a robust language model such as BERT can enhance the performance. Using a more robust classifier also enhances the effect of the language model on the answer selection component. The results demonstrate that language comprehension is an essential requirement in natural language processing tasks such as answer-selection.
Tasks	Answer Selection, Language Modelling, Question Answering
Published	2019-11-04
URL	https://arxiv.org/abs/1911.01528v3
PDF	https://arxiv.org/pdf/1911.01528v3.pdf
PWC	https://paperswithcode.com/paper/bas-an-answer-selection-method-using-bert
Repo
Framework

An Explicitly Relational Neural Network Architecture


Title	An Explicitly Relational Neural Network Architecture
Authors	Murray Shanahan, Kyriacos Nikiforou, Antonia Creswell, Christos Kaplanis, David Barrett, Marta Garnelo
Abstract	With a view to bridging the gap between deep learning and symbolic AI, we present a novel end-to-end neural network architecture that learns to form propositional representations with an explicitly relational structure from raw pixel data. In order to evaluate and analyse the architecture, we introduce a family of simple visual relational reasoning tasks of varying complexity. We show that the proposed architecture, when pre-trained on a curriculum of such tasks, learns to generate reusable representations that better facilitate subsequent learning on previously unseen tasks when compared to a number of baseline architectures. The workings of a successfully trained model are visualised to shed some light on how the architecture functions.
Tasks	Relational Reasoning
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10307v3
PDF	https://arxiv.org/pdf/1905.10307v3.pdf
PWC	https://paperswithcode.com/paper/an-explicitly-relational-neural-network
Repo
Framework

Generating Question-Answer Hierarchies


Title	Generating Question-Answer Hierarchies
Authors	Kalpesh Krishna, Mohit Iyyer
Abstract	The process of knowledge acquisition can be viewed as a question-answer game between a student and a teacher in which the student typically starts by asking broad, open-ended questions before drilling down into specifics (Hintikka, 1981; Hakkarainen and Sintonen, 2002). This pedagogical perspective motivates a new way of representing documents. In this paper, we present SQUASH (Specificity-controlled Question-Answer Hierarchies), a novel and challenging text generation task that converts an input document into a hierarchy of question-answer pairs. Users can click on high-level questions (e.g., “Why did Frodo leave the Fellowship?") to reveal related but more specific questions (e.g., “Who did Frodo leave with?"). Using a question taxonomy loosely based on Lehnert (1978), we classify questions in existing reading comprehension datasets as either “general” or “specific”. We then use these labels as input to a pipelined system centered around a conditional neural language model. We extensively evaluate the quality of the generated QA hierarchies through crowdsourced experiments and report strong empirical results.
Tasks	Language Modelling, Reading Comprehension, Text Generation
Published	2019-06-06
URL	https://arxiv.org/abs/1906.02622v2
PDF	https://arxiv.org/pdf/1906.02622v2.pdf
PWC	https://paperswithcode.com/paper/generating-question-answer-hierarchies
Repo
Framework

Attentive Representation Learning with Adversarial Training for Short Text Clustering


Title	Attentive Representation Learning with Adversarial Training for Short Text Clustering
Authors	Wei Zhang, Chao Dong, Jianhua Yin, Jianyong Wang
Abstract	Short text clustering has far-reaching effects on semantic analysis, showing its importance for multiple applications such as corpus summarization and information retrieval. However, it inevitably encounters the severe sparsity of short text representation, making the previous clustering approaches still far from satisfactory. In this paper, we present a novel attentive representation learning model for shot text clustering, wherein cluster-level attention is proposed to capture the correlation between text representation and cluster representation. Relying on this, the representation learning and clustering for short text are seamlessly integrated into a unified framework. To further facilitate the model training process, we apply adversarial training to the unsupervised clustering setting, by adding perturbations to the cluster representations. The model parameters and perturbations are optimized alternately through a minimax game. Extensive experiments on three real-world short text datasets demonstrate the superiority of the proposed model over several strong competitors, verifying that adversarial training yields a substantial performance gain.
Tasks	Information Retrieval, Representation Learning, Text Clustering
Published	2019-12-08
URL	https://arxiv.org/abs/1912.03720v1
PDF	https://arxiv.org/pdf/1912.03720v1.pdf
PWC	https://paperswithcode.com/paper/attentive-representation-learning-with
Repo
Framework

Text Mining using Nonnegative Matrix Factorization and Latent Semantic Analysis


Title	Text Mining using Nonnegative Matrix Factorization and Latent Semantic Analysis
Authors	Ali Hassani, Amir Iranmanesh, Najme Mansouri
Abstract	Text clustering is arguably one of the most important topics in modern data mining. Nevertheless, text data require tokenization which usually yields a very large and highly sparse term-document matrix, which is usually difficult to process using conventional machine learning algorithms. Methods such as Latent Semantic Analysis have helped mitigate this issue, but are nevertheless not completely stable in practice. As a result, we propose a new feature agglomeration method based on Nonnegative Matrix Factorization, which is employed to separate the terms into groups, and then each group’s term vectors are agglomerated into a new feature vector. Together, these feature vectors create a new feature space much more suitable for clustering. In addition, we propose a new deterministic initialization for spherical K-Means, which proves very useful for this specific type of data. In order to evaluate the proposed method, we compare it to some of the latest research done in this field, as well as some of the most practiced methods. In our experiments, we conclude that the proposed method either significantly improves clustering performance, or maintains the performance of other methods, while improving stability in results.
Tasks	Text Clustering, Tokenization
Published	2019-11-12
URL	https://arxiv.org/abs/1911.04705v3
PDF	https://arxiv.org/pdf/1911.04705v3.pdf
PWC	https://paperswithcode.com/paper/text-mining-using-nonnegative-matrix
Repo
Framework

Low Rank Approximation for Smoothing Spline via Eigensystem Truncation


Title	Low Rank Approximation for Smoothing Spline via Eigensystem Truncation
Authors	Danqing Xu, Yuedong Wang
Abstract	Smoothing splines provide a powerful and flexible means for nonparametric estimation and inference. With a cubic time complexity, fitting smoothing spline models to large data is computationally prohibitive. In this paper, we use the theoretical optimal eigenspace to derive a low rank approximation of the smoothing spline estimates. We develop a method to approximate the eigensystem when it is unknown and derive error bounds for the approximate estimates. The proposed methods are easy to implement with existing software. Extensive simulations show that the new methods are accurate, fast, and compares favorably against existing methods.
Tasks
Published	2019-11-23
URL	https://arxiv.org/abs/1911.10434v1
PDF	https://arxiv.org/pdf/1911.10434v1.pdf
PWC	https://paperswithcode.com/paper/low-rank-approximation-for-smoothing-spline
Repo
Framework

An Autoencoder-based Learned Image Compressor: Description of Challenge Proposal by NCTU


Title	An Autoencoder-based Learned Image Compressor: Description of Challenge Proposal by NCTU
Authors	David Alexandre, Chih-Peng Chang, Wen-Hsiao Peng, Hsueh-Ming Hang
Abstract	We propose a lossy image compression system using the deep-learning autoencoder structure to participate in the Challenge on Learned Image Compression (CLIC) 2018. Our autoencoder uses the residual blocks with skip connections to reduce the correlation among image pixels and condense the input image into a set of feature maps, a compact representation of the original image. The bit allocation and bitrate control are implemented by using the importance maps and quantizer. The importance maps are generated by a separate neural net in the encoder. The autoencoder and the importance net are trained jointly based on minimizing a weighted sum of mean squared error, MS-SSIM, and a rate estimate. Our aim is to produce reconstructed images with good subjective quality subject to the 0.15 bits-per-pixel constraint.
Tasks	Image Compression
Published	2019-02-20
URL	http://arxiv.org/abs/1902.07385v1
PDF	http://arxiv.org/pdf/1902.07385v1.pdf
PWC	https://paperswithcode.com/paper/an-autoencoder-based-learned-image-compressor
Repo
Framework

Do Human Rationales Improve Machine Explanations?


Title	Do Human Rationales Improve Machine Explanations?
Authors	Julia Strout, Ye Zhang, Raymond J. Mooney
Abstract	Work on “learning with rationales” shows that humans providing explanations to a machine learning system can improve the system’s predictive accuracy. However, this work has not been connected to work in “explainable AI” which concerns machines explaining their reasoning to humans. In this work, we show that learning with rationales can also improve the quality of the machine’s explanations as evaluated by human judges. Specifically, we present experiments showing that, for CNN- based text classification, explanations generated using “supervised attention” are judged superior to explanations generated using normal unsupervised attention.
Tasks	Text Classification
Published	2019-05-31
URL	https://arxiv.org/abs/1905.13714v1
PDF	https://arxiv.org/pdf/1905.13714v1.pdf
PWC	https://paperswithcode.com/paper/do-human-rationales-improve-machine
Repo
Framework

An end-to-end Neural Network Framework for Text Clustering


Title	An end-to-end Neural Network Framework for Text Clustering
Authors	Jie Zhou, Xingyi Cheng, Jinchao Zhang
Abstract	The unsupervised text clustering is one of the major tasks in natural language processing (NLP) and remains a difficult and complex problem. Conventional \mbox{methods} generally treat this task using separated steps, including text representation learning and clustering the representations. As an improvement, neural methods have also been introduced for continuous representation learning to address the sparsity problem. However, the multi-step process still deviates from the unified optimization target. Especially the second step of cluster is generally performed with conventional methods such as k-Means. We propose a pure neural framework for text clustering in an end-to-end manner. It jointly learns the text representation and the clustering model. Our model works well when the context can be obtained, which is nearly always the case in the field of NLP. We have our method \mbox{evaluated} on two widely used benchmarks: IMDB movie reviews for sentiment classification and $20$-Newsgroup for topic categorization. Despite its simplicity, experiments show the model outperforms previous clustering methods by a large margin. Furthermore, the model is also verified on English wiki dataset as a large corpus.
Tasks	Representation Learning, Sentiment Analysis, Text Clustering
Published	2019-03-22
URL	http://arxiv.org/abs/1903.09424v1
PDF	http://arxiv.org/pdf/1903.09424v1.pdf
PWC	https://paperswithcode.com/paper/an-end-to-end-neural-network-framework-for
Repo
Framework