October 17, 2019

2748 words 13 mins read

Paper Group ANR 910

Semantic Variation in Online Communities of Practice. A Constrained Randomized Shortest-Paths Framework for Optimal Exploration. Degeneration in VAE: in the Light of Fisher Information Loss. Persistence pays off: Paying Attention to What the LSTM Gating Mechanism Persists. Improving Neural Question Generation using Answer Separation. Relating Eye-T …

Semantic Variation in Online Communities of Practice


Title	Semantic Variation in Online Communities of Practice
Authors	Marco Del Tredici, Raquel Fernández
Abstract	We introduce a framework for quantifying semantic variation of common words in Communities of Practice and in sets of topic-related communities. We show that while some meaning shifts are shared across related communities, others are community-specific, and therefore independent from the discussed topic. We propose such findings as evidence in favour of sociolinguistic theories of socially-driven semantic variation. Results are evaluated using an independent language modelling task. Furthermore, we investigate extralinguistic features and show that factors such as prominence and dissemination of words are related to semantic variation.
Tasks	Language Modelling
Published	2018-06-15
URL	http://arxiv.org/abs/1806.05847v1
PDF	http://arxiv.org/pdf/1806.05847v1.pdf
PWC	https://paperswithcode.com/paper/semantic-variation-in-online-communities-of
Repo
Framework

A Constrained Randomized Shortest-Paths Framework for Optimal Exploration


Title	A Constrained Randomized Shortest-Paths Framework for Optimal Exploration
Authors	Bertrand Lebichot, Guillaume Guex, Ilkka Kivimäki, Marco Saerens
Abstract	The present work extends the randomized shortest-paths framework (RSP), interpolating between shortest-path and random-walk routing in a network, in three directions. First, it shows how to deal with equality constraints on a subset of transition probabilities and develops a generic algorithm for solving this constrained RSP problem using Lagrangian duality. Second, it derives a surprisingly simple iterative procedure to compute the optimal, randomized, routing policy generalizing the previously developed “soft” Bellman-Ford algorithm. The resulting algorithm allows balancing exploitation and exploration in an optimal way by interpolating between a pure random behavior and the deterministic, optimal, policy (least-cost paths) while satisfying the constraints. Finally, the two algorithms are applied to Markov decision problems by considering the process as a constrained RSP on a bipartite state-action graph. In this context, the derived “soft” value iteration algorithm appears to be closely related to dynamic policy programming as well as Kullback-Leibler and path integral control, and similar to a recently introduced reinforcement learning exploration strategy. This shows that this strategy is optimal in the RSP sense - it minimizes expected path cost subject to relative entropy constraint. Simulation results on illustrative examples show that the model behaves as expected.
Tasks
Published	2018-07-12
URL	http://arxiv.org/abs/1807.04551v1
PDF	http://arxiv.org/pdf/1807.04551v1.pdf
PWC	https://paperswithcode.com/paper/a-constrained-randomized-shortest-paths
Repo
Framework

Degeneration in VAE: in the Light of Fisher Information Loss


Title	Degeneration in VAE: in the Light of Fisher Information Loss
Authors	Huangjie Zheng, Jiangchao Yao, Ya Zhang, Ivor W. Tsang
Abstract	While enormous progress has been made to Variational Autoencoder (VAE) in recent years, similar to other deep networks, VAE with deep networks suffers from the problem of degeneration, which seriously weakens the correlation between the input and the corresponding latent codes, deviating from the goal of the representation learning. To investigate how degeneration affects VAE from a theoretical perspective, we illustrate the information transmission in VAE and analyze the intermediate layers of the encoders/decoders. Specifically, we propose a Fisher Information measure for the layer-wise analysis. With such measure, we demonstrate that information loss is ineluctable in feed-forward networks and causes the degeneration in VAE. We show that skip connections in VAE enable the preservation of information without changing the model architecture. We call this class of VAE equipped with skip connections as SCVAE and perform a range of experiments to show its advantages in information preservation and degeneration mitigation.
Tasks	Representation Learning
Published	2018-02-19
URL	http://arxiv.org/abs/1802.06677v3
PDF	http://arxiv.org/pdf/1802.06677v3.pdf
PWC	https://paperswithcode.com/paper/degeneration-in-vae-in-the-light-of-fisher
Repo
Framework

Persistence pays off: Paying Attention to What the LSTM Gating Mechanism Persists


Title	Persistence pays off: Paying Attention to What the LSTM Gating Mechanism Persists
Authors	Giancarlo D. Salton, John D. Kelleher
Abstract	Language Models (LMs) are important components in several Natural Language Processing systems. Recurrent Neural Network LMs composed of LSTM units, especially those augmented with an external memory, have achieved state-of-the-art results. However, these models still struggle to process long sequences which are more likely to contain long-distance dependencies because of information fading and a bias towards more recent information. In this paper we demonstrate an effective mechanism for retrieving information in a memory augmented LSTM LM based on attending to information in memory in proportion to the number of timesteps the LSTM gating mechanism persisted the information.
Tasks
Published	2018-10-10
URL	http://arxiv.org/abs/1810.04437v1
PDF	http://arxiv.org/pdf/1810.04437v1.pdf
PWC	https://paperswithcode.com/paper/persistence-pays-off-paying-attention-to-what
Repo
Framework

Improving Neural Question Generation using Answer Separation


Title	Improving Neural Question Generation using Answer Separation
Authors	Yanghoon Kim, Hwanhee Lee, Joongbo Shin, Kyomin Jung
Abstract	Neural question generation (NQG) is the task of generating a question from a given passage with deep neural networks. Previous NQG models suffer from a problem that a significant proportion of the generated questions include words in the question target, resulting in the generation of unintended questions. In this paper, we propose answer-separated seq2seq, which better utilizes the information from both the passage and the target answer. By replacing the target answer in the original passage with a special token, our model learns to identify which interrogative word should be used. We also propose a new module termed keyword-net, which helps the model better capture the key information in the target answer and generate an appropriate question. Experimental results demonstrate that our answer separation method significantly reduces the number of improper questions which include answers. Consequently, our model significantly outperforms previous state-of-the-art NQG models.
Tasks	Question Generation
Published	2018-09-07
URL	http://arxiv.org/abs/1809.02393v2
PDF	http://arxiv.org/pdf/1809.02393v2.pdf
PWC	https://paperswithcode.com/paper/improving-neural-question-generation-using
Repo
Framework

Relating Eye-Tracking Measures With Changes In Knowledge on Search Tasks


Title	Relating Eye-Tracking Measures With Changes In Knowledge on Search Tasks
Authors	Nilavra Bhattacharya, Jacek Gwizdka
Abstract	We conducted an eye-tracking study where 30 participants performed searches on the web. We measured their topical knowledge before and after each task. Their eye-fixations were labelled as “reading” or “scanning”. The series of reading fixations in a line, called “reading-sequences” were characterized by their length in pixels, fixation duration, and the number of fixations making up the sequence. We hypothesize that differences in knowledge-change of participants are reflected in their eye-tracking measures related to reading. Our results show that the participants with higher change in knowledge differ significantly in terms of their total reading-sequence-length, reading-sequence-duration, and number of reading fixations, when compared to participants with lower knowledge-change.
Tasks	Eye Tracking
Published	2018-05-07
URL	http://arxiv.org/abs/1805.02399v1
PDF	http://arxiv.org/pdf/1805.02399v1.pdf
PWC	https://paperswithcode.com/paper/relating-eye-tracking-measures-with-changes
Repo
Framework

Tools for higher-order network analysis


Title	Tools for higher-order network analysis
Authors	Austin R. Benson
Abstract	Networks are a fundamental model of complex systems throughout the sciences, and network datasets are typically analyzed through lower-order connectivity patterns described at the level of individual nodes and edges. However, higher-order connectivity patterns captured by small subgraphs, also called network motifs, describe the fundamental structures that control and mediate the behavior of many complex systems. We develop three tools for network analysis that use higher-order connectivity patterns to gain new insights into network datasets: (1) a framework to cluster nodes into modules based on joint participation in network motifs; (2) a generalization of the clustering coefficient measurement to investigate higher-order closure patterns; and (3) a definition of network motifs for temporal networks and fast algorithms for counting them. Using these tools, we analyze data from biology, ecology, economics, neuroscience, online social networks, scientific collaborations, telecommunications, transportation, and the World Wide Web.
Tasks
Published	2018-02-19
URL	http://arxiv.org/abs/1802.06820v1
PDF	http://arxiv.org/pdf/1802.06820v1.pdf
PWC	https://paperswithcode.com/paper/tools-for-higher-order-network-analysis
Repo
Framework

Pseudo-marginal Bayesian inference for supervised Gaussian process latent variable models


Title	Pseudo-marginal Bayesian inference for supervised Gaussian process latent variable models
Authors	Charles Gadd, Sara Wade, Akeel Shah, Dimitris Grammatopoulos
Abstract	We introduce a Bayesian framework for inference with a supervised version of the Gaussian process latent variable model. The framework overcomes the high correlations between latent variables and hyperparameters by using an unbiased pseudo estimate for the marginal likelihood that approximately integrates over the latent variables. This is used to construct a Markov Chain to explore the posterior of the hyperparameters. We demonstrate the procedure on simulated and real examples, showing its ability to capture uncertainty and multimodality of the hyperparameters and improved uncertainty quantification in predictions when compared with variational inference.
Tasks	Bayesian Inference, Latent Variable Models
Published	2018-03-28
URL	http://arxiv.org/abs/1803.10746v1
PDF	http://arxiv.org/pdf/1803.10746v1.pdf
PWC	https://paperswithcode.com/paper/pseudo-marginal-bayesian-inference-for
Repo
Framework

Cancer Research UK Drug Discovery Process Mining


Title	Cancer Research UK Drug Discovery Process Mining
Authors	Haochao Huang
Abstract	Background. The Drug Discovery Unit (DDU) of Cancer Research UK (CRUK) is using the software Dotmatics for storage and analysis of scientific data during drug discovery process. Whilst the data include event logs, time stamps, activities, and user information are mostly sitting in the database without fully utilising their potential value. Aims. This dissertation aims at extracting knowledge from event logs data which recorded during drug discovery process, to capture the operational business process of the DDU of Cancer Research UK (CRUK) as it was being executed. It provides the evaluations and methodologies of drawing the process mining panoramic models for the drug discovery process. Thus by enabling the DDU to maximise its efficiency in reviewing its resources and works allocations, patients will benefit from more new treatments faster. Conclusion. Management of organisations can be benefit from the process mining methodologies. Disco is excellent for non-experts on management purposes. ProM is great for expert on research purposes. However, the process mining is not once and for all but is a regular operation management process. Indeed, event logs needs to be understand more on the target organisational behaviours and organisational business process. The researchers have to be aware that event logs data are the most important and priority elements in process mining.
Tasks	Drug Discovery
Published	2018-05-18
URL	http://arxiv.org/abs/1805.08169v1
PDF	http://arxiv.org/pdf/1805.08169v1.pdf
PWC	https://paperswithcode.com/paper/cancer-research-uk-drug-discovery-process
Repo
Framework

Comment on: Decomposition of structural learning about directed acyclic graphs [1]


Title	Comment on: Decomposition of structural learning about directed acyclic graphs [1]
Authors	Mohammad Ali Javidian, Marco Valtorta
Abstract	We propose an alternative proof concerning necessary and sufficient conditions to split the problem of searching for d-separators and building the skeleton of a DAG into small problems for every node of a separation tree T. The proof is simpler than the original [1]. The same proof structure has been used in [2] for learning the structure of multivariate regression chain graphs (MVR CGs).
Tasks
Published	2018-06-27
URL	http://arxiv.org/abs/1806.11103v1
PDF	http://arxiv.org/pdf/1806.11103v1.pdf
PWC	https://paperswithcode.com/paper/comment-on-decomposition-of-structural
Repo
Framework

Tackling Multilabel Imbalance through Label Decoupling and Data Resampling Hybridization


Title	Tackling Multilabel Imbalance through Label Decoupling and Data Resampling Hybridization
Authors	Francisco Charte, Antonio J. Rivera, María J. del Jesus, Francisco Herrera
Abstract	The learning from imbalanced data is a deeply studied problem in standard classification and, in recent times, also in multilabel classification. A handful of multilabel resampling methods have been proposed in late years, aiming to balance the labels distribution. However these methods have to face a new obstacle, specific for multilabel data, as is the joint appearance of minority and majority labels in the same data patterns. We proposed recently a new algorithm designed to decouple imbalanced labels concurring in the same instance, called REMEDIAL (\textit{REsampling MultilabEl datasets by Decoupling highly ImbAlanced Labels}). The goal of this work is to propose a procedure to hybridize this method with some of the best resampling algorithms available in the literature, including random oversampling, heuristic undersampling and synthetic sample generation techniques. These hybrid methods are then empirically analyzed, determining how their behavior is influenced by the label decoupling process. As a result, a noteworthy set of guidelines on the combined use of these techniques can be drawn from the conducted experimentation.
Tasks
Published	2018-02-14
URL	http://arxiv.org/abs/1802.05031v1
PDF	http://arxiv.org/pdf/1802.05031v1.pdf
PWC	https://paperswithcode.com/paper/tackling-multilabel-imbalance-through-label
Repo
Framework

Head Reconstruction from Internet Photos


Title	Head Reconstruction from Internet Photos
Authors	Shu Liang, Linda G. Shapiro, Ira Kemelmacher-Shlizerman
Abstract	3D face reconstruction from Internet photos has recently produced exciting results. A person’s face, e.g., Tom Hanks, can be modeled and animated in 3D from a completely uncalibrated photo collection. Most methods, however, focus solely on face area and mask out the rest of the head. This paper proposes that head modeling from the Internet is a problem we can solve. We target reconstruction of the rough shape of the head. Our method is to gradually “grow” the head mesh starting from the frontal face and extending to the rest of views using photometric stereo constraints. We call our method boundary-value growing algorithm. Results on photos of celebrities downloaded from the Internet are presented.
Tasks	3D Face Reconstruction, Face Reconstruction
Published	2018-09-13
URL	http://arxiv.org/abs/1809.04763v1
PDF	http://arxiv.org/pdf/1809.04763v1.pdf
PWC	https://paperswithcode.com/paper/head-reconstruction-from-internet-photos
Repo
Framework

Affective EEG-Based Person Identification Using the Deep Learning Approach


Title	Affective EEG-Based Person Identification Using the Deep Learning Approach
Authors	Theerawit Wilaiprasitporn, Apiwat Ditthapron, Karis Matchaparn, Tanaboon Tongbuasirilai, Nannapas Banluesombatkul, Ekapol Chuangsuwanich
Abstract	Electroencephalography (EEG) is another mode for performing Person Identification (PI). Due to the nature of the EEG signals, EEG-based PI is typically done while the person is performing some kind of mental task, such as motor control. However, few works have considered EEG-based PI while the person is in different mental states (affective EEG). The aim of this paper is to improve the performance of affective EEG-based PI using a deep learning approach. \textcolor{red}{We proposed a cascade of deep learning using a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs)}. CNNs are used to handle the spatial information from the EEG while RNNs extract the temporal information. \textcolor{red}{We evaluated two types of RNNs, namely, Long Short-Term Memory (CNN-LSTM) and Gated Recurrent Unit (CNN-GRU). } The proposed method is evaluated on the state-of-the-art affective dataset DEAP. The results indicate that CNN-GRU and CNN-LSTM can perform PI from different affective states and reach up to 99.90–100% mean Correct Recognition Rate (CRR), significantly outperforming a support vector machine (SVM) baseline system that uses power spectral density (PSD) features. Notably, the 100% mean \emph{CRR} comes from only 40 subjects in DEAP dataset. To reduce the number of EEG electrodes from thirty-two to five for more practical applications, the frontal region gives the best results reaching up to 99.17% CRR (from CNN-GRU). Amongst the two deep learning models, we find CNN-GRU to slightly outperform CNN-LSTM, while having faster training time. \textcolor{red}{Furthermore, CNN-GRU overcomes the influence of affective states in EEG-Based PI reported in the previous works.
Tasks	EEG, Person Identification
Published	2018-07-05
URL	http://arxiv.org/abs/1807.03147v3
PDF	http://arxiv.org/pdf/1807.03147v3.pdf
PWC	https://paperswithcode.com/paper/affective-eeg-based-person-identification
Repo
Framework

Conformal Prediction in Learning Under Privileged Information Paradigm with Applications in Drug Discovery


Title	Conformal Prediction in Learning Under Privileged Information Paradigm with Applications in Drug Discovery
Authors	Niharika Gauraha, Lars Carlsson, Ola Spjuth
Abstract	This paper explores conformal prediction in the learning under privileged information (LUPI) paradigm. We use the SVM+ realization of LUPI in an inductive conformal predictor, and apply it to the MNIST benchmark dataset and three datasets in drug discovery. The results show that using privileged information produces valid models and improves efficiency compared to standard SVM, however the improvement varies between the tested datasets and is not substantial in the drug discovery applications. More importantly, using SVM+ in a conformal prediction framework enables valid prediction intervals at specified significance levels.
Tasks	Drug Discovery
Published	2018-03-29
URL	http://arxiv.org/abs/1803.11136v2
PDF	http://arxiv.org/pdf/1803.11136v2.pdf
PWC	https://paperswithcode.com/paper/conformal-prediction-in-learning-under
Repo
Framework

A Comprehensive Survey for Low Rank Regularization


Title	A Comprehensive Survey for Low Rank Regularization
Authors	Zhanxuan Hu, Feiping Nie, Lai Tian, Rong Wang, Xuelong Li
Abstract	Low rank regularization, in essence, involves introducing a low rank or approximately low rank assumption for matrix we aim to learn, which has achieved great success in many fields including machine learning, data mining and computer version. Over the last decade, much progress has been made in theories and practical applications. Nevertheless, the intersection between them is very slight. In order to construct a bridge between practical applications and theoretical research, in this paper we provide a comprehensive survey for low rank regularization. We first review several traditional machine learning models using low rank regularization, and then show their (or their variants) applications in solving practical issues, such as non-rigid structure from motion and image denoising. Subsequently, we summarize the regularizers and optimization methods that achieve great success in traditional machine learning tasks but are rarely seen in solving practical issues. Finally, we provide a discussion and comparison for some representative regularizers including convex and non-convex relaxations. Extensive experimental results demonstrate that non-convex regularizers can provide a large advantage over the nuclear norm, the regularizer widely used in solving practical issues.
Tasks	Denoising, Image Denoising
Published	2018-08-14
URL	http://arxiv.org/abs/1808.04521v2
PDF	http://arxiv.org/pdf/1808.04521v2.pdf
PWC	https://paperswithcode.com/paper/a-comprehensive-survey-for-low-rank
Repo
Framework