July 27, 2019

3330 words 16 mins read

Paper Group ANR 655

Analysis and Optimization of fastText Linear Text Classifier. Mining Frequent Patterns in Process Models. Sound-Word2Vec: Learning Word Representations Grounded in Sounds. Diffusion geometry unravels the emergence of functional clusters in collective phenomena. Learning Algorithms for Active Learning. Personalized Survival Predictions for Cardiac T …

Analysis and Optimization of fastText Linear Text Classifier


Title	Analysis and Optimization of fastText Linear Text Classifier
Authors	Vladimir Zolotov, David Kung
Abstract	The paper [1] shows that simple linear classifier can compete with complex deep learning algorithms in text classification applications. Combining bag of words (BoW) and linear classification techniques, fastText [1] attains same or only slightly lower accuracy than deep learning algorithms [2-9] that are orders of magnitude slower. We proved formally that fastText can be transformed into a simpler equivalent classifier, which unlike fastText does not have any hidden layer. We also proved that the necessary and sufficient dimensionality of the word vector embedding space is exactly the number of document classes. These results help constructing more optimal linear text classifiers with guaranteed maximum classification capabilities. The results are proven exactly by pure formal algebraic methods without attracting any empirical data.
Tasks	Text Classification
Published	2017-02-17
URL	http://arxiv.org/abs/1702.05531v1
PDF	http://arxiv.org/pdf/1702.05531v1.pdf
PWC	https://paperswithcode.com/paper/analysis-and-optimization-of-fasttext-linear
Repo
Framework

Mining Frequent Patterns in Process Models


Title	Mining Frequent Patterns in Process Models
Authors	David Chapela-Campa, Manuel Mucientes, Manuel Lama
Abstract	Process mining has emerged as a way to analyze the behavior of an organization by extracting knowledge from event logs and by offering techniques to discover, monitor and enhance real processes. In the discovery of process models, retrieving a complex one, i.e., a hardly readable process model, can hinder the extraction of information. Even in well-structured process models, there is information that cannot be obtained with the current techniques. In this paper, we present WoMine, an algorithm to retrieve frequent behavioural patterns from the model. Our approach searches in process models extracting structures with sequences, selections, parallels and loops, which are frequently executed in the logs. This proposal has been validated with a set of process models, including some from BPI Challenges, and compared with the state of the art techniques. Experiments have validated that WoMine can find all types of patterns, extracting information that cannot be mined with the state of the art techniques.
Tasks
Published	2017-10-11
URL	http://arxiv.org/abs/1710.05693v1
PDF	http://arxiv.org/pdf/1710.05693v1.pdf
PWC	https://paperswithcode.com/paper/mining-frequent-patterns-in-process-models
Repo
Framework

Sound-Word2Vec: Learning Word Representations Grounded in Sounds


Title	Sound-Word2Vec: Learning Word Representations Grounded in Sounds
Authors	Ashwin K Vijayakumar, Ramakrishna Vedantam, Devi Parikh
Abstract	To be able to interact better with humans, it is crucial for machines to understand sound - a primary modality of human perception. Previous works have used sound to learn embeddings for improved generic textual similarity assessment. In this work, we treat sound as a first-class citizen, studying downstream textual tasks which require aural grounding. To this end, we propose sound-word2vec - a new embedding scheme that learns specialized word embeddings grounded in sounds. For example, we learn that two seemingly (semantically) unrelated concepts, like leaves and paper are similar due to the similar rustling sounds they make. Our embeddings prove useful in textual tasks requiring aural reasoning like text-based sound retrieval and discovering foley sound effects (used in movies). Moreover, our embedding space captures interesting dependencies between words and onomatopoeia and outperforms prior work on aurally-relevant word relatedness datasets such as AMEN and ASLex.
Tasks	Word Embeddings
Published	2017-03-06
URL	http://arxiv.org/abs/1703.01720v4
PDF	http://arxiv.org/pdf/1703.01720v4.pdf
PWC	https://paperswithcode.com/paper/sound-word2vec-learning-word-representations
Repo
Framework

Diffusion geometry unravels the emergence of functional clusters in collective phenomena


Title	Diffusion geometry unravels the emergence of functional clusters in collective phenomena
Authors	Manlio De Domenico
Abstract	Collective phenomena emerge from the interaction of natural or artificial units with a complex organization. The interplay between structural patterns and dynamics might induce functional clusters that, in general, are different from topological ones. In biological systems, like the human brain, the overall functionality is often favored by the interplay between connectivity and synchronization dynamics, with functional clusters that do not coincide with anatomical modules in most cases. In social, socio-technical and engineering systems, the quest for consensus favors the emergence of clusters. Despite the unquestionable evidence for mesoscale organization of many complex systems and the heterogeneity of their inter-connectivity, a way to predict and identify the emergence of functional modules in collective phenomena continues to elude us. Here, we propose an approach based on random walk dynamics to define the diffusion distance between any pair of units in a networked system. Such a metric allows to exploit the underlying diffusion geometry to provide a unifying framework for the intimate relationship between metastable synchronization, consensus and random search dynamics in complex networks, pinpointing the functional mesoscale organization of synthetic and biological systems.
Tasks
Published	2017-04-24
URL	http://arxiv.org/abs/1704.07068v1
PDF	http://arxiv.org/pdf/1704.07068v1.pdf
PWC	https://paperswithcode.com/paper/diffusion-geometry-unravels-the-emergence-of
Repo
Framework

Learning Algorithms for Active Learning


Title	Learning Algorithms for Active Learning
Authors	Philip Bachman, Alessandro Sordoni, Adam Trischler
Abstract	We introduce a model that learns active learning algorithms via metalearning. For a distribution of related tasks, our model jointly learns: a data representation, an item selection heuristic, and a method for constructing prediction functions from labeled training sets. Our model uses the item selection heuristic to gather labeled training sets from which to construct prediction functions. Using the Omniglot and MovieLens datasets, we test our model in synthetic and practical settings.
Tasks	Active Learning, Omniglot
Published	2017-07-31
URL	http://arxiv.org/abs/1708.00088v1
PDF	http://arxiv.org/pdf/1708.00088v1.pdf
PWC	https://paperswithcode.com/paper/learning-algorithms-for-active-learning
Repo
Framework

Personalized Survival Predictions for Cardiac Transplantation via Trees of Predictors


Title	Personalized Survival Predictions for Cardiac Transplantation via Trees of Predictors
Authors	J. Yoon, W. R. Zame, A. Banerjee, M. Cadeiras, A. M. Alaa, M. van der Schaar
Abstract	Given the limited pool of donor organs, accurate predictions of survival on the wait list and post transplantation are crucial for cardiac transplantation decisions and policy. However, current clinical risk scores do not yield accurate predictions. We develop a new methodology (ToPs, Trees of Predictors) built on the principle that specific predictors should be used for specific clusters within the target population. ToPs discovers these specific clusters of patients and the specific predictor that perform best for each cluster. In comparison with current clinical risk scoring systems, our method provides significant improvements in the prediction of survival time on the wait list and post transplantation. For example, in terms of 3 month survival for patients who were on the US patient wait list in the period 1985 to 2015, our method achieves AUC of 0.847, the best commonly used clinical risk score (MAGGIC) achieves 0.630. In terms of 3 month survival/mortality predictions (in comparison to MAGGIC), holding specificity at 80.0 percents, our algorithm correctly predicts survival for 1,228 (26.0 percents more patients out of 4,723 who actually survived, holding sensitivity at 80.0 percents, our algorithm correctly predicts mortality for 839 (33.0 percents) more patients out of 2,542 who did not survive. Our method achieves similar improvements for other time horizons and for predictions post transplantation. Therefore, we offer a more accurate, personalized approach to survival analysis that can benefit patients, clinicians and policymakers in making clinical decisions and setting clinical policy. Because risk prediction is widely used in diagnostic and prognostic clinical decision making across diseases and clinical specialties, the implications of our methods are far reaching.
Tasks	Decision Making, Survival Analysis
Published	2017-04-11
URL	http://arxiv.org/abs/1704.03458v1
PDF	http://arxiv.org/pdf/1704.03458v1.pdf
PWC	https://paperswithcode.com/paper/personalized-survival-predictions-for-cardiac
Repo
Framework

Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity


Title	Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity
Authors	Jack Hessel, Lillian Lee, David Mimno
Abstract	The content of today’s social media is becoming more and more rich, increasingly mixing text, images, videos, and audio. It is an intriguing research question to model the interplay between these different modes in attracting user attention and engagement. But in order to pursue this study of multimodal content, we must also account for context: timing effects, community preferences, and social factors (e.g., which authors are already popular) also affect the amount of feedback and reaction that social-media posts receive. In this work, we separate out the influence of these non-content factors in several ways. First, we focus on ranking pairs of submissions posted to the same community in quick succession, e.g., within 30 seconds, this framing encourages models to focus on time-agnostic and community-specific content features. Within that setting, we determine the relative performance of author vs. content features. We find that victory usually belongs to “cats and captions,” as visual and textual features together tend to outperform identity-based features. Moreover, our experiments show that when considered in isolation, simple unigram text features and deep neural network visual features yield the highest accuracy individually, and that the combination of the two modalities generally leads to the best accuracies overall.
Tasks
Published	2017-03-06
URL	http://arxiv.org/abs/1703.01725v1
PDF	http://arxiv.org/pdf/1703.01725v1.pdf
PWC	https://paperswithcode.com/paper/cats-and-captions-vs-creators-and-the-clock
Repo
Framework

Bayesian Joint Matrix Decomposition for Data Integration with Heterogeneous Noise


Title	Bayesian Joint Matrix Decomposition for Data Integration with Heterogeneous Noise
Authors	Chihao Zhang, Shihua Zhang
Abstract	Matrix decomposition is a popular and fundamental approach in machine learning and data mining. It has been successfully applied into various fields. Most matrix decomposition methods focus on decomposing a data matrix from one single source. However, it is common that data are from different sources with heterogeneous noise. A few of matrix decomposition methods have been extended for such multi-view data integration and pattern discovery. While only few methods were designed to consider the heterogeneity of noise in such multi-view data for data integration explicitly. To this end, we propose a joint matrix decomposition framework (BJMD), which models the heterogeneity of noise by Gaussian distribution in a Bayesian framework. We develop two algorithms to solve this model: one is a variational Bayesian inference algorithm, which makes full use of the posterior distribution; and another is a maximum a posterior algorithm, which is more scalable and can be easily paralleled. Extensive experiments on synthetic and real-world datasets demonstrate that BJMD considering the heterogeneity of noise is superior or competitive to the state-of-the-art methods.
Tasks	Bayesian Inference
Published	2017-12-09
URL	http://arxiv.org/abs/1712.03337v1
PDF	http://arxiv.org/pdf/1712.03337v1.pdf
PWC	https://paperswithcode.com/paper/bayesian-joint-matrix-decomposition-for-data
Repo
Framework

Tensor Balancing on Statistical Manifold


Title	Tensor Balancing on Statistical Manifold
Authors	Mahito Sugiyama, Hiroyuki Nakahara, Koji Tsuda
Abstract	We solve tensor balancing, rescaling an Nth order nonnegative tensor by multiplying N tensors of order N - 1 so that every fiber sums to one. This generalizes a fundamental process of matrix balancing used to compare matrices in a wide range of applications from biology to economics. We present an efficient balancing algorithm with quadratic convergence using Newton’s method and show in numerical experiments that the proposed algorithm is several orders of magnitude faster than existing ones. To theoretically prove the correctness of the algorithm, we model tensors as probability distributions in a statistical manifold and realize tensor balancing as projection onto a submanifold. The key to our algorithm is that the gradient of the manifold, used as a Jacobian matrix in Newton’s method, can be analytically obtained using the Moebius inversion formula, the essential of combinatorial mathematics. Our model is not limited to tensor balancing, but has a wide applicability as it includes various statistical and machine learning models such as weighted DAGs and Boltzmann machines.
Tasks
Published	2017-02-27
URL	http://arxiv.org/abs/1702.08142v3
PDF	http://arxiv.org/pdf/1702.08142v3.pdf
PWC	https://paperswithcode.com/paper/tensor-balancing-on-statistical-manifold
Repo
Framework

A Novel Comprehensive Approach for Estimating Concept Semantic Similarity in WordNet


Title	A Novel Comprehensive Approach for Estimating Concept Semantic Similarity in WordNet
Authors	Xiao-gang Zhang, Shou-qian Sun, Ke-jun Zhang
Abstract	Computation of semantic similarity between concepts is an important foundation for many research works. This paper focuses on IC computing methods and IC measures, which estimate the semantic similarities between concepts by exploiting the topological parameters of the taxonomy. Based on analyzing representative IC computing methods and typical semantic similarity measures, we propose a new hybrid IC computing method. Through adopting the parameter dhyp and lch, we utilize the new IC computing method and propose a novel comprehensive measure of semantic similarity between concepts. An experiment based on WordNet “is a” taxonomy has been designed to test representative measures and our measure on benchmark dataset R&G, and the results show that our measure can obviously improve the similarity accuracy. We evaluate the proposed approach by comparing the correlation coefficients between five measures and the artificial data. The results show that our proposal outperforms the previous measures.
Tasks	Semantic Similarity, Semantic Textual Similarity
Published	2017-03-06
URL	http://arxiv.org/abs/1703.01726v1
PDF	http://arxiv.org/pdf/1703.01726v1.pdf
PWC	https://paperswithcode.com/paper/a-novel-comprehensive-approach-for-estimating
Repo
Framework

Fast kNN mode seeking clustering applied to active learning


Title	Fast kNN mode seeking clustering applied to active learning
Authors	Robert P. W. Duin, Sergey Verzakov
Abstract	A significantly faster algorithm is presented for the original kNN mode seeking procedure. It has the advantages over the well-known mean shift algorithm that it is feasible in high-dimensional vector spaces and results in uniquely, well defined modes. Moreover, without any additional computational effort it may yield a multi-scale hierarchy of clusterings. The time complexity is just O(n^1.5). resulting computing times range from seconds for 10^4 objects to minutes for 10^5 objects and to less than an hour for 10^6 objects. The space complexity is just O(n). The procedure is well suited for finding large sets of small clusters and is thereby a candidate to analyze thousands of clusters in millions of objects. The kNN mode seeking procedure can be used for active learning by assigning the clusters to the class of the modal objects of the clusters. Its feasibility is shown by some examples with up to 1.5 million handwritten digits. The obtained classification results based on the clusterings are compared with those obtained by the nearest neighbor rule and the support vector classifier based on the same labeled objects for training. It can be concluded that using the clustering structure for classification can be significantly better than using the trained classifiers. A drawback of using the clustering for classification, however, is that no classifier is obtained that may be used for out-of-sample objects.
Tasks	Active Learning
Published	2017-12-20
URL	http://arxiv.org/abs/1712.07454v1
PDF	http://arxiv.org/pdf/1712.07454v1.pdf
PWC	https://paperswithcode.com/paper/fast-knn-mode-seeking-clustering-applied-to
Repo
Framework

Statistical learning of spatiotemporal patterns from longitudinal manifold-valued networks


Title	Statistical learning of spatiotemporal patterns from longitudinal manifold-valued networks
Authors	Igor Koval, Jean-Baptiste Schiratti, Alexandre Routier, Michael Bacci, Olivier Colliot, Stéphanie Allassonnière, Stanley Durrleman
Abstract	We introduce a mixed-effects model to learn spatiotempo-ral patterns on a network by considering longitudinal measures distributed on a fixed graph. The data come from repeated observations of subjects at different time points which take the form of measurement maps distributed on a graph such as an image or a mesh. The model learns a typical group-average trajectory characterizing the propagation of measurement changes across the graph nodes. The subject-specific trajectories are defined via spatial and temporal transformations of the group-average scenario, thus estimating the variability of spatiotemporal patterns within the group. To estimate population and individual model parameters, we adapted a stochastic version of the Expectation-Maximization algorithm, the MCMC-SAEM. The model is used to describe the propagation of cortical atrophy during the course of Alzheimer’s Disease. Model parameters show the variability of this average pattern of atrophy in terms of trajectories across brain regions, age at disease onset and pace of propagation. We show that the personalization of this model yields accurate prediction of maps of cortical thickness in patients.
Tasks
Published	2017-09-25
URL	http://arxiv.org/abs/1709.08491v1
PDF	http://arxiv.org/pdf/1709.08491v1.pdf
PWC	https://paperswithcode.com/paper/statistical-learning-of-spatiotemporal
Repo
Framework

A computational investigation of sources of variability in sentence comprehension difficulty in aphasia


Title	A computational investigation of sources of variability in sentence comprehension difficulty in aphasia
Authors	Paul Mätzig, Shravan Vasishth, Felix Engelmann, David Caplan
Abstract	We present a computational evaluation of three hypotheses about sources of deficit in sentence comprehension in aphasia: slowed processing, intermittent deficiency, and resource reduction. The ACT-R based Lewis and Vasishth (2005) model is used to implement these three proposals. Slowed processing is implemented as slowed default production-rule firing time; intermittent deficiency as increased random noise in activation of chunks in memory; and resource reduction as reduced goal activation. As data, we considered subject vs. object rela- tives whose matrix clause contained either an NP or a reflexive, presented in a self-paced listening modality to 56 individuals with aphasia (IWA) and 46 matched controls. The participants heard the sentences and carried out a picture verification task to decide on an interpretation of the sentence. These response accuracies are used to identify the best parameters (for each participant) that correspond to the three hypotheses mentioned above. We show that controls have more tightly clustered (less variable) parameter values than IWA; specifically, compared to controls, among IWA there are more individuals with low goal activations, high noise, and slow default action times. This suggests that (i) individual patients show differential amounts of deficit along the three dimensions of slowed processing, intermittent deficient, and resource reduction, (ii) overall, there is evidence for all three sources of deficit playing a role, and (iii) IWA have a more variable range of parameter values than controls. In sum, this study contributes a proof of concept of a quantitative implementation of, and evidence for, these three accounts of comprehension deficits in aphasia.
Tasks
Published	2017-03-14
URL	http://arxiv.org/abs/1703.04677v2
PDF	http://arxiv.org/pdf/1703.04677v2.pdf
PWC	https://paperswithcode.com/paper/a-computational-investigation-of-sources-of
Repo
Framework

Summarization of User-Generated Sports Video by Using Deep Action Recognition Features


Title	Summarization of User-Generated Sports Video by Using Deep Action Recognition Features
Authors	Antonio Tejero-de-Pablos, Yuta Nakashima, Tomokazu Sato, Naokazu Yokoya, Marko Linna, Esa Rahtu
Abstract	Automatically generating a summary of sports video poses the challenge of detecting interesting moments, or highlights, of a game. Traditional sports video summarization methods leverage editing conventions of broadcast sports video that facilitate the extraction of high-level semantics. However, user-generated videos are not edited, and thus traditional methods are not suitable to generate a summary. In order to solve this problem, this work proposes a novel video summarization method that uses players’ actions as a cue to determine the highlights of the original video. A deep neural network-based approach is used to extract two types of action-related features and to classify video segments into interesting or uninteresting parts. The proposed method can be applied to any sports in which games consist of a succession of actions. Especially, this work considers the case of Kendo (Japanese fencing) as an example of a sport to evaluate the proposed method. The method is trained using Kendo videos with ground truth labels that indicate the video highlights. The labels are provided by annotators possessing different experience with respect to Kendo to demonstrate how the proposed method adapts to different needs. The performance of the proposed method is compared with several combinations of different features, and the results show that it outperforms previous summarization methods.
Tasks	Temporal Action Localization, Video Summarization
Published	2017-09-25
URL	http://arxiv.org/abs/1709.08421v2
PDF	http://arxiv.org/pdf/1709.08421v2.pdf
PWC	https://paperswithcode.com/paper/summarization-of-user-generated-sports-video
Repo
Framework

Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning


Title	Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning
Authors	Joshua Achiam, Shankar Sastry
Abstract	Exploration in complex domains is a key challenge in reinforcement learning, especially for tasks with very sparse rewards. Recent successes in deep reinforcement learning have been achieved mostly using simple heuristic exploration strategies such as $\epsilon$-greedy action selection or Gaussian control noise, but there are many tasks where these methods are insufficient to make any learning progress. Here, we consider more complex heuristics: efficient and scalable exploration strategies that maximize a notion of an agent’s surprise about its experiences via intrinsic motivation. We propose to learn a model of the MDP transition probabilities concurrently with the policy, and to form intrinsic rewards that approximate the KL-divergence of the true transition probabilities from the learned model. One of our approximations results in using surprisal as intrinsic motivation, while the other gives the $k$-step learning progress. We show that our incentives enable agents to succeed in a wide range of environments with high-dimensional state spaces and very sparse rewards, including continuous control tasks and games in the Atari RAM domain, outperforming several other heuristic exploration techniques.
Tasks	Continuous Control
Published	2017-03-06
URL	http://arxiv.org/abs/1703.01732v1
PDF	http://arxiv.org/pdf/1703.01732v1.pdf
PWC	https://paperswithcode.com/paper/surprise-based-intrinsic-motivation-for-deep
Repo
Framework