Paper Group ANR 419
Improving Multi-label Learning with Missing Labels by Structured Semantic Correlations. Regulating Reward Training by Means of Certainty Prediction in a Neural Network-Implemented Pong Game. Comparative study of LSA vs Word2vec embeddings in small corpora: a case study in dreams database. Knowledge-Based Biomedical Word Sense Disambiguation with Ne …
Improving Multi-label Learning with Missing Labels by Structured Semantic Correlations
| Title | Improving Multi-label Learning with Missing Labels by Structured Semantic Correlations |
| Authors | Hao Yang, Joey Tianyi Zhou, Jianfei Cai |
| Abstract | Multi-label learning has attracted significant interests in computer vision recently, finding applications in many vision tasks such as multiple object recognition and automatic image annotation. Associating multiple labels to a complex image is very difficult, not only due to the intricacy of describing the image, but also because of the incompleteness nature of the observed labels. Existing works on the problem either ignore the label-label and instance-instance correlations or just assume these correlations are linear and unstructured. Considering that semantic correlations between images are actually structured, in this paper we propose to incorporate structured semantic correlations to solve the missing label problem of multi-label learning. Specifically, we project images to the semantic space with an effective semantic descriptor. A semantic graph is then constructed on these images to capture the structured correlations between them. We utilize the semantic graph Laplacian as a smooth term in the multi-label learning formulation to incorporate the structured semantic correlations. Experimental results demonstrate the effectiveness of the proposed semantic descriptor and the usefulness of incorporating the structured semantic correlations. We achieve better results than state-of-the-art multi-label learning methods on four benchmark datasets. |
| Tasks | Multi-Label Learning, Object Recognition |
| Published | 2016-08-04 |
| URL | http://arxiv.org/abs/1608.01441v1 |
| http://arxiv.org/pdf/1608.01441v1.pdf | |
| PWC | https://paperswithcode.com/paper/improving-multi-label-learning-with-missing |
| Repo | |
| Framework | |
Regulating Reward Training by Means of Certainty Prediction in a Neural Network-Implemented Pong Game
| Title | Regulating Reward Training by Means of Certainty Prediction in a Neural Network-Implemented Pong Game |
| Authors | Matt Oberdorfer, Matt Abuzalaf |
| Abstract | We present the first reinforcement-learning model to self-improve its reward-modulated training implemented through a continuously improving “intuition” neural network. An agent was trained how to play the arcade video game Pong with two reward-based alternatives, one where the paddle was placed randomly during training, and a second where the paddle was simultaneously trained on three additional neural networks such that it could develop a sense of “certainty” as to how probable its own predicted paddle position will be to return the ball. If the agent was less than 95% certain to return the ball, the policy used an intuition neural network to place the paddle. We trained both architectures for an equivalent number of epochs and tested learning performance by letting the trained programs play against a near-perfect opponent. Through this, we found that the reinforcement learning model that uses an intuition neural network for placing the paddle during reward training quickly overtakes the simple architecture in its ability to outplay the near-perfect opponent, additionally outscoring that opponent by an increasingly wide margin after additional epochs of training. |
| Tasks | |
| Published | 2016-09-23 |
| URL | http://arxiv.org/abs/1609.07434v1 |
| http://arxiv.org/pdf/1609.07434v1.pdf | |
| PWC | https://paperswithcode.com/paper/regulating-reward-training-by-means-of |
| Repo | |
| Framework | |
Comparative study of LSA vs Word2vec embeddings in small corpora: a case study in dreams database
| Title | Comparative study of LSA vs Word2vec embeddings in small corpora: a case study in dreams database |
| Authors | Edgar Altszyler, Mariano Sigman, Sidarta Ribeiro, Diego Fernández Slezak |
| Abstract | Word embeddings have been extensively studied in large text datasets. However, only a few studies analyze semantic representations of small corpora, particularly relevant in single-person text production studies. In the present paper, we compare Skip-gram and LSA capabilities in this scenario, and we test both techniques to extract relevant semantic patterns in single-series dreams reports. LSA showed better performance than Skip-gram in small size training corpus in two semantic tests. As a study case, we show that LSA can capture relevant words associations in dream reports series, even in cases of small number of dreams or low-frequency words. We propose that LSA can be used to explore words associations in dreams reports, which could bring new insight into this classic research area of psychology |
| Tasks | Word Embeddings |
| Published | 2016-10-05 |
| URL | http://arxiv.org/abs/1610.01520v2 |
| http://arxiv.org/pdf/1610.01520v2.pdf | |
| PWC | https://paperswithcode.com/paper/comparative-study-of-lsa-vs-word2vec |
| Repo | |
| Framework | |
Knowledge-Based Biomedical Word Sense Disambiguation with Neural Concept Embeddings
| Title | Knowledge-Based Biomedical Word Sense Disambiguation with Neural Concept Embeddings |
| Authors | A. K. M. Sabbir, Antonio Jimeno Yepes, Ramakanth Kavuluru |
| Abstract | Biomedical word sense disambiguation (WSD) is an important intermediate task in many natural language processing applications such as named entity recognition, syntactic parsing, and relation extraction. In this paper, we employ knowledge-based approaches that also exploit recent advances in neural word/concept embeddings to improve over the state-of-the-art in biomedical WSD using the MSH WSD dataset as the test set. Our methods involve weak supervision - we do not use any hand-labeled examples for WSD to build our prediction models; however, we employ an existing well known named entity recognition and concept mapping program, MetaMap, to obtain our concept vectors. Over the MSH WSD dataset, our linear time (in terms of numbers of senses and words in the test instance) method achieves an accuracy of 92.24% which is an absolute 3% improvement over the best known results obtained via unsupervised or knowledge-based means. A more expensive approach that we developed relies on a nearest neighbor framework and achieves an accuracy of 94.34%. Employing dense vector representations learned from unlabeled free text has been shown to benefit many language processing tasks recently and our efforts show that biomedical WSD is no exception to this trend. For a complex and rapidly evolving domain such as biomedicine, building labeled datasets for larger sets of ambiguous terms may be impractical. Here, we show that weak supervision that leverages recent advances in representation learning can rival supervised approaches in biomedical WSD. However, external knowledge bases (here sense inventories) play a key role in the improvements achieved. |
| Tasks | Named Entity Recognition, Relation Extraction, Representation Learning, Word Sense Disambiguation |
| Published | 2016-10-26 |
| URL | http://arxiv.org/abs/1610.08557v5 |
| http://arxiv.org/pdf/1610.08557v5.pdf | |
| PWC | https://paperswithcode.com/paper/knowledge-based-biomedical-word-sense |
| Repo | |
| Framework | |
Onsager-corrected deep learning for sparse linear inverse problems
| Title | Onsager-corrected deep learning for sparse linear inverse problems |
| Authors | Mark Borgerding, Philip Schniter |
| Abstract | Deep learning has gained great popularity due to its widespread success on many inference problems. We consider the application of deep learning to the sparse linear inverse problem encountered in compressive sensing, where one seeks to recover a sparse signal from a small number of noisy linear measurements. In this paper, we propose a novel neural-network architecture that decouples prediction errors across layers in the same way that the approximate message passing (AMP) algorithm decouples them across iterations: through Onsager correction. Numerical experiments suggest that our “learned AMP” network significantly improves upon Gregor and LeCun’s “learned ISTA” network in both accuracy and complexity. |
| Tasks | Compressive Sensing |
| Published | 2016-07-20 |
| URL | http://arxiv.org/abs/1607.05966v1 |
| http://arxiv.org/pdf/1607.05966v1.pdf | |
| PWC | https://paperswithcode.com/paper/onsager-corrected-deep-learning-for-sparse |
| Repo | |
| Framework | |
Manifold-Kernels Comparison in MKPLS for Visual Speech Recognition
| Title | Manifold-Kernels Comparison in MKPLS for Visual Speech Recognition |
| Authors | Amr Bakry, Ahmed Elgammal |
| Abstract | Speech recognition is a challenging problem. Due to the acoustic limitations, using visual information is essential for improving the recognition accuracy in real-life unconstraint situations. One common approach is to model the visual recognition as nonlinear optimization problem. Measuring the distances between visual units is essential for solving this problem. Embedding the visual units on a manifold and using manifold kernels is one way to measure these distances. This work is intended to evaluate the performance of several manifold kernels for solving the problem of visual speech recognition. We show the theory behind each kernel. We apply manifold kernel partial least squares framework to OuluVs and AvLetters databases, and show empirical comparison between all kernels. This framework provides convenient way to explore different kernels. |
| Tasks | Speech Recognition, Visual Speech Recognition |
| Published | 2016-01-22 |
| URL | http://arxiv.org/abs/1601.05861v1 |
| http://arxiv.org/pdf/1601.05861v1.pdf | |
| PWC | https://paperswithcode.com/paper/manifold-kernels-comparison-in-mkpls-for |
| Repo | |
| Framework | |
DARI: Distance metric And Representation Integration for Person Verification
| Title | DARI: Distance metric And Representation Integration for Person Verification |
| Authors | Guangrun Wang, Liang Lin, Shengyong Ding, Ya Li, Qing Wang |
| Abstract | The past decade has witnessed the rapid development of feature representation learning and distance metric learning, whereas the two steps are often discussed separately. To explore their interaction, this work proposes an end-to-end learning framework called DARI, i.e. Distance metric And Representation Integration, and validates the effectiveness of DARI in the challenging task of person verification. Given the training images annotated with the labels, we first produce a large number of triplet units, and each one contains three images, i.e. one person and the matched/mismatch references. For each triplet unit, the distance disparity between the matched pair and the mismatched pair tends to be maximized. We solve this objective by building a deep architecture of convolutional neural networks. In particular, the Mahalanobis distance matrix is naturally factorized as one top fully-connected layer that is seamlessly integrated with other bottom layers representing the image feature. The image feature and the distance metric can be thus simultaneously optimized via the one-shot backward propagation. On several public datasets, DARI shows very promising performance on re-identifying individuals cross cameras against various challenges, and outperforms other state-of-the-art approaches. |
| Tasks | Metric Learning, Representation Learning |
| Published | 2016-04-15 |
| URL | http://arxiv.org/abs/1604.04377v1 |
| http://arxiv.org/pdf/1604.04377v1.pdf | |
| PWC | https://paperswithcode.com/paper/dari-distance-metric-and-representation |
| Repo | |
| Framework | |
Scalable Influence Maximization for Multiple Products in Continuous-Time Diffusion Networks
| Title | Scalable Influence Maximization for Multiple Products in Continuous-Time Diffusion Networks |
| Authors | Nan Du, Yingyu Liang, Maria-Florina Balcan, Manuel Gomez-Rodriguez, Hongyuan Zha, Le Song |
| Abstract | A typical viral marketing model identifies influential users in a social network to maximize a single product adoption assuming unlimited user attention, campaign budgets, and time. In reality, multiple products need campaigns, users have limited attention, convincing users incurs costs, and advertisers have limited budgets and expect the adoptions to be maximized soon. Facing these user, monetary, and timing constraints, we formulate the problem as a submodular maximization task in a continuous-time diffusion model under the intersection of a matroid and multiple knapsack constraints. We propose a randomized algorithm estimating the user influence in a network ($\mathcal{V}$ nodes, $\mathcal{E}$ edges) to an accuracy of $\epsilon$ with $n=\mathcal{O}(1/\epsilon^2)$ randomizations and $\tilde{\mathcal{O}}(n\mathcal{E}+n\mathcal{V})$ computations. By exploiting the influence estimation algorithm as a subroutine, we develop an adaptive threshold greedy algorithm achieving an approximation factor $k_a/(2+2 k)$ of the optimal when $k_a$ out of the $k$ knapsack constraints are active. Extensive experiments on networks of millions of nodes demonstrate that the proposed algorithms achieve the state-of-the-art in terms of effectiveness and scalability. |
| Tasks | |
| Published | 2016-12-08 |
| URL | http://arxiv.org/abs/1612.02712v2 |
| http://arxiv.org/pdf/1612.02712v2.pdf | |
| PWC | https://paperswithcode.com/paper/scalable-influence-maximization-for-multiple |
| Repo | |
| Framework | |
One-Pass Learning with Incremental and Decremental Features
| Title | One-Pass Learning with Incremental and Decremental Features |
| Authors | Chenping Hou, Zhi-Hua Zhou |
| Abstract | In many real tasks the features are evolving, with some features being vanished and some other features augmented. For example, in environment monitoring some sensors expired whereas some new ones deployed; in mobile game recommendation some games dropped whereas some new ones added. Learning with such incremental and decremental features is crucial but rarely studied, particularly when the data coming like a stream and thus it is infeasible to keep the whole data for optimization. In this paper, we study this challenging problem and present the OPID approach. Our approach attempts to compress important information of vanished features into functions of survived features, and then expand to include the augmented features. It is the one-pass learning approach, which only needs to scan each instance once and does not need to store the whole data, and thus satisfy the evolving streaming data nature. The effectiveness of our approach is validated theoretically and empirically. |
| Tasks | |
| Published | 2016-05-30 |
| URL | http://arxiv.org/abs/1605.09082v1 |
| http://arxiv.org/pdf/1605.09082v1.pdf | |
| PWC | https://paperswithcode.com/paper/one-pass-learning-with-incremental-and |
| Repo | |
| Framework | |
Visualizing and Understanding Sum-Product Networks
| Title | Visualizing and Understanding Sum-Product Networks |
| Authors | Antonio Vergari, Nicola Di Mauro, Floriana Esposito |
| Abstract | Sum-Product Networks (SPNs) are recently introduced deep tractable probabilistic models by which several kinds of inference queries can be answered exactly and in a tractable time. Up to now, they have been largely used as black box density estimators, assessed only by comparing their likelihood scores only. In this paper we explore and exploit the inner representations learned by SPNs. We do this with a threefold aim: first we want to get a better understanding of the inner workings of SPNs; secondly, we seek additional ways to evaluate one SPN model and compare it against other probabilistic models, providing diagnostic tools to practitioners; lastly, we want to empirically evaluate how good and meaningful the extracted representations are, as in a classic Representation Learning framework. In order to do so we revise their interpretation as deep neural networks and we propose to exploit several visualization techniques on their node activations and network outputs under different types of inference queries. To investigate these models as feature extractors, we plug some SPNs, learned in a greedy unsupervised fashion on image datasets, in supervised classification learning tasks. We extract several embedding types from node activations by filtering nodes by their type, by their associated feature abstraction level and by their scope. In a thorough empirical comparison we prove them to be competitive against those generated from popular feature extractors as Restricted Boltzmann Machines. Finally, we investigate embeddings generated from random probabilistic marginal queries as means to compare other tractable probabilistic models on a common ground, extending our experiments to Mixtures of Trees. |
| Tasks | Representation Learning |
| Published | 2016-08-29 |
| URL | http://arxiv.org/abs/1608.08266v2 |
| http://arxiv.org/pdf/1608.08266v2.pdf | |
| PWC | https://paperswithcode.com/paper/visualizing-and-understanding-sum-product |
| Repo | |
| Framework | |
On Stochastic Belief Revision and Update and their Combination
| Title | On Stochastic Belief Revision and Update and their Combination |
| Authors | Gavin Rens |
| Abstract | I propose a framework for an agent to change its probabilistic beliefs when a new piece of propositional information $\alpha$ is observed. Traditionally, belief change occurs by either a revision process or by an update process, depending on whether the agent is informed with $\alpha$ in a static world or, respectively, whether $\alpha$ is a ‘signal’ from the environment due to an event occurring. Boutilier suggested a unified model of qualitative belief change, which “combines aspects of revision and update, providing a more realistic characterization of belief change.” In this paper, I propose a unified model of quantitative belief change, where an agent’s beliefs are represented as a probability distribution over possible worlds. As does Boutilier, I take a dynamical systems perspective. The proposed approach is evaluated against several rationality postulated, and some properties of the approach are worked out. |
| Tasks | |
| Published | 2016-04-07 |
| URL | http://arxiv.org/abs/1604.02126v1 |
| http://arxiv.org/pdf/1604.02126v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-stochastic-belief-revision-and-update-and |
| Repo | |
| Framework | |
Horn: A System for Parallel Training and Regularizing of Large-Scale Neural Networks
| Title | Horn: A System for Parallel Training and Regularizing of Large-Scale Neural Networks |
| Authors | Edward J. Yoon |
| Abstract | I introduce a new distributed system for effective training and regularizing of Large-Scale Neural Networks on distributed computing architectures. The experiments demonstrate the effectiveness of flexible model partitioning and parallelization strategies based on neuron-centric computation model, with an implementation of the collective and parallel dropout neural networks training. Experiments are performed on MNIST handwritten digits classification including results. |
| Tasks | |
| Published | 2016-08-02 |
| URL | http://arxiv.org/abs/1608.00781v2 |
| http://arxiv.org/pdf/1608.00781v2.pdf | |
| PWC | https://paperswithcode.com/paper/horn-a-system-for-parallel-training-and |
| Repo | |
| Framework | |
Measuring Machine Intelligence Through Visual Question Answering
| Title | Measuring Machine Intelligence Through Visual Question Answering |
| Authors | C. Lawrence Zitnick, Aishwarya Agrawal, Stanislaw Antol, Margaret Mitchell, Dhruv Batra, Devi Parikh |
| Abstract | As machines have become more intelligent, there has been a renewed interest in methods for measuring their intelligence. A common approach is to propose tasks for which a human excels, but one which machines find difficult. However, an ideal task should also be easy to evaluate and not be easily gameable. We begin with a case study exploring the recently popular task of image captioning and its limitations as a task for measuring machine intelligence. An alternative and more promising task is Visual Question Answering that tests a machine’s ability to reason about language and vision. We describe a dataset unprecedented in size created for the task that contains over 760,000 human generated questions about images. Using around 10 million human generated answers, machines may be easily evaluated. |
| Tasks | Image Captioning, Question Answering, Visual Question Answering |
| Published | 2016-08-31 |
| URL | http://arxiv.org/abs/1608.08716v1 |
| http://arxiv.org/pdf/1608.08716v1.pdf | |
| PWC | https://paperswithcode.com/paper/measuring-machine-intelligence-through-visual |
| Repo | |
| Framework | |
An empirical study on large scale text classification with skip-gram embeddings
| Title | An empirical study on large scale text classification with skip-gram embeddings |
| Authors | Georgios Balikas, Massih-Reza Amini |
| Abstract | We investigate the integration of word embeddings as classification features in the setting of large scale text classification. Such representations have been used in a plethora of tasks, however their application in classification scenarios with thousands of classes has not been extensively researched, partially due to hardware limitations. In this work, we examine efficient composition functions to obtain document-level from word-level embeddings and we subsequently investigate their combination with the traditional one-hot-encoding representations. By presenting empirical evidence on large, multi-class, multi-label classification problems, we demonstrate the efficiency and the performance benefits of this combination. |
| Tasks | Multi-Label Classification, Text Classification, Word Embeddings |
| Published | 2016-06-21 |
| URL | http://arxiv.org/abs/1606.06623v1 |
| http://arxiv.org/pdf/1606.06623v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-empirical-study-on-large-scale-text |
| Repo | |
| Framework | |
Learning Interpretability for Visualizations using Adapted Cox Models through a User Experiment
| Title | Learning Interpretability for Visualizations using Adapted Cox Models through a User Experiment |
| Authors | Adrien Bibal, Benoit Frénay |
| Abstract | In order to be useful, visualizations need to be interpretable. This paper uses a user-based approach to combine and assess quality measures in order to better model user preferences. Results show that cluster separability measures are outperformed by a neighborhood conservation measure, even though the former are usually considered as intuitively representative of user motives. Moreover, combining measures, as opposed to using a single measure, further improves prediction performances. |
| Tasks | |
| Published | 2016-11-18 |
| URL | http://arxiv.org/abs/1611.06175v1 |
| http://arxiv.org/pdf/1611.06175v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-interpretability-for-visualizations |
| Repo | |
| Framework | |
