Paper Group ANR 118
One-Shot Learning in Discriminative Neural Networks. A Deep Learning Approach for Population Estimation from Satellite Imagery. Parallel and in-process compilation of individuals for genetic programming on GPU. The principle of cognitive action - Preliminary experimental analysis. Improving precision and recall of face recognition in SIPP with comb …
One-Shot Learning in Discriminative Neural Networks
| Title | One-Shot Learning in Discriminative Neural Networks |
| Authors | Jordan Burgess, James Robert Lloyd, Zoubin Ghahramani |
| Abstract | We consider the task of one-shot learning of visual categories. In this paper we explore a Bayesian procedure for updating a pretrained convnet to classify a novel image category for which data is limited. We decompose this convnet into a fixed feature extractor and softmax classifier. We assume that the target weights for the new task come from the same distribution as the pretrained softmax weights, which we model as a multivariate Gaussian. By using this as a prior for the new weights, we demonstrate competitive performance with state-of-the-art methods whilst also being consistent with ‘normal’ methods for training deep networks on large data. |
| Tasks | One-Shot Learning |
| Published | 2017-07-18 |
| URL | http://arxiv.org/abs/1707.05562v1 |
| http://arxiv.org/pdf/1707.05562v1.pdf | |
| PWC | https://paperswithcode.com/paper/one-shot-learning-in-discriminative-neural |
| Repo | |
| Framework | |
A Deep Learning Approach for Population Estimation from Satellite Imagery
| Title | A Deep Learning Approach for Population Estimation from Satellite Imagery |
| Authors | Caleb Robinson, Fred Hohman, Bistra Dilkina |
| Abstract | Knowing where people live is a fundamental component of many decision making processes such as urban development, infectious disease containment, evacuation planning, risk management, conservation planning, and more. While bottom-up, survey driven censuses can provide a comprehensive view into the population landscape of a country, they are expensive to realize, are infrequently performed, and only provide population counts over broad areas. Population disaggregation techniques and population projection methods individually address these shortcomings, but also have shortcomings of their own. To jointly answer the questions of “where do people live” and “how many people live there,” we propose a deep learning model for creating high-resolution population estimations from satellite imagery. Specifically, we train convolutional neural networks to predict population in the USA at a $0.01^{\circ} \times 0.01^{\circ}$ resolution grid from 1-year composite Landsat imagery. We validate these models in two ways: quantitatively, by comparing our model’s grid cell estimates aggregated at a county-level to several US Census county-level population projections, and qualitatively, by directly interpreting the model’s predictions in terms of the satellite image inputs. We find that aggregating our model’s estimates gives comparable results to the Census county-level population projections and that the predictions made by our model can be directly interpreted, which give it advantages over traditional population disaggregation methods. In general, our model is an example of how machine learning techniques can be an effective tool for extracting information from inherently unstructured, remotely sensed data to provide effective solutions to social problems. |
| Tasks | Decision Making |
| Published | 2017-08-30 |
| URL | http://arxiv.org/abs/1708.09086v1 |
| http://arxiv.org/pdf/1708.09086v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-deep-learning-approach-for-population |
| Repo | |
| Framework | |
Parallel and in-process compilation of individuals for genetic programming on GPU
| Title | Parallel and in-process compilation of individuals for genetic programming on GPU |
| Authors | Hakan Ayral, Songül Albayrak |
| Abstract | Three approaches to implement genetic programming on GPU hardware are compilation, interpretation and direct generation of machine code. The compiled approach is known to have a prohibitive overhead compared to other two. This paper investigates methods to accelerate compilation of individuals for genetic programming on GPU hardware. We apply in-process compilation to minimize the compilation overhead at each generation; and we investigate ways to parallelize in-process compilation. In-process compilation doesn’t lend itself to trivial parallelization with threads; we propose a multiprocess parallelization using memory sharing and operating systems interprocess communication primitives. With parallelized compilation we achieve further reductions on compilation overhead. Another contribution of this work is the code framework we built in C# for the experiments. The framework makes it possible to build arbitrary grammatical genetic programming experiments that run on GPU with minimal extra coding effort, and is available as open source. |
| Tasks | |
| Published | 2017-05-21 |
| URL | http://arxiv.org/abs/1705.07492v1 |
| http://arxiv.org/pdf/1705.07492v1.pdf | |
| PWC | https://paperswithcode.com/paper/parallel-and-in-process-compilation-of |
| Repo | |
| Framework | |
The principle of cognitive action - Preliminary experimental analysis
| Title | The principle of cognitive action - Preliminary experimental analysis |
| Authors | Marco Gori, Marco Maggini, Alessandro Rossi |
| Abstract | In this document we shows a first implementation and some preliminary results of a new theory, facing Machine Learning problems in the frameworks of Classical Mechanics and Variational Calculus. We give a general formulation of the problem and then we studies basic behaviors of the model on simple practical implementations. |
| Tasks | |
| Published | 2017-01-09 |
| URL | http://arxiv.org/abs/1701.02377v1 |
| http://arxiv.org/pdf/1701.02377v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-principle-of-cognitive-action-preliminary |
| Repo | |
| Framework | |
Improving precision and recall of face recognition in SIPP with combination of modified mean search and LSH
| Title | Improving precision and recall of face recognition in SIPP with combination of modified mean search and LSH |
| Authors | Xihua Li |
| Abstract | Although face recognition has been improved much as the development of Deep Neural Networks, SIPP(Single Image Per Person) problem in face recognition has not been better solved, especially in practical applications where searching over complicated database. In this paper, a combination of modified mean search and LSH method would be introduced orderly to improve the precision and recall of SIPP face recognition without retrain of the DNN model. First, a modified SVD based augmentation method would be introduced to get more intra-class variations even for person with only one image. Second, an unique rule based combination of modified mean search and LSH method was proposed the first time to help get the most similar personID in a complicated dataset, and some theoretical explaining followed. Third, we would like to emphasize, no need to retrain of the DNN model and would easy to be extended without much efforts. We do some practical testing in competition of Msceleb challenge-2 2017 which was hold by Microsoft Research, great improvement of coverage from 13.39% to 19.25%, 29.94%, 42.11%, 47.52% at precision 99%(P99) would be shown latter, coverage reach 94.2% and 100% at precision 97%(P97) and 95%(P95) respectively. As far as we known, this is the only paper who do not fine-tuning on competition dataset and ranked top-10. A similar test on CASIA WebFace dataset also demonstrated the same improvements on both precision and recall. |
| Tasks | Face Recognition |
| Published | 2017-09-09 |
| URL | http://arxiv.org/abs/1709.03872v2 |
| http://arxiv.org/pdf/1709.03872v2.pdf | |
| PWC | https://paperswithcode.com/paper/improving-precision-and-recall-of-face |
| Repo | |
| Framework | |
Gradient Diversity: a Key Ingredient for Scalable Distributed Learning
| Title | Gradient Diversity: a Key Ingredient for Scalable Distributed Learning |
| Authors | Dong Yin, Ashwin Pananjady, Max Lam, Dimitris Papailiopoulos, Kannan Ramchandran, Peter Bartlett |
| Abstract | It has been experimentally observed that distributed implementations of mini-batch stochastic gradient descent (SGD) algorithms exhibit speedup saturation and decaying generalization ability beyond a particular batch-size. In this work, we present an analysis hinting that high similarity between concurrently processed gradients may be a cause of this performance degradation. We introduce the notion of gradient diversity that measures the dissimilarity between concurrent gradient updates, and show its key role in the performance of mini-batch SGD. We prove that on problems with high gradient diversity, mini-batch SGD is amenable to better speedups, while maintaining the generalization performance of serial (one sample) SGD. We further establish lower bounds on convergence where mini-batch SGD slows down beyond a particular batch-size, solely due to the lack of gradient diversity. We provide experimental evidence indicating the key role of gradient diversity in distributed learning, and discuss how heuristics like dropout, Langevin dynamics, and quantization can improve it. |
| Tasks | Quantization |
| Published | 2017-06-18 |
| URL | http://arxiv.org/abs/1706.05699v3 |
| http://arxiv.org/pdf/1706.05699v3.pdf | |
| PWC | https://paperswithcode.com/paper/gradient-diversity-a-key-ingredient-for |
| Repo | |
| Framework | |
Learning in High-Dimensional Multimedia Data: The State of the Art
| Title | Learning in High-Dimensional Multimedia Data: The State of the Art |
| Authors | Lianli Gao, Jingkuan Song, Xingyi Liu, Junming Shao, Jiajun Liu, Jie Shao |
| Abstract | During the last decade, the deluge of multimedia data has impacted a wide range of research areas, including multimedia retrieval, 3D tracking, database management, data mining, machine learning, social media analysis, medical imaging, and so on. Machine learning is largely involved in multimedia applications of building models for classification and regression tasks etc., and the learning principle consists in designing the models based on the information contained in the multimedia dataset. While many paradigms exist and are widely used in the context of machine learning, most of them suffer from the `curse of dimensionality’, which means that some strange phenomena appears when data are represented in a high-dimensional space. Given the high dimensionality and the high complexity of multimedia data, it is important to investigate new machine learning algorithms to facilitate multimedia data analysis. To deal with the impact of high dimensionality, an intuitive way is to reduce the dimensionality. On the other hand, some researchers devoted themselves to designing some effective learning schemes for high-dimensional data. In this survey, we cover feature transformation, feature selection and feature encoding, three approaches fighting the consequences of the curse of dimensionality. Next, we briefly introduce some recent progress of effective learning algorithms. Finally, promising future trends on multimedia learning are envisaged. | |
| Tasks | Feature Selection |
| Published | 2017-07-10 |
| URL | http://arxiv.org/abs/1707.02683v1 |
| http://arxiv.org/pdf/1707.02683v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-in-high-dimensional-multimedia-data |
| Repo | |
| Framework | |
AMPNet: Asynchronous Model-Parallel Training for Dynamic Neural Networks
| Title | AMPNet: Asynchronous Model-Parallel Training for Dynamic Neural Networks |
| Authors | Alexander L. Gaunt, Matthew A. Johnson, Maik Riechert, Daniel Tarlow, Ryota Tomioka, Dimitrios Vytiniotis, Sam Webster |
| Abstract | New types of machine learning hardware in development and entering the market hold the promise of revolutionizing deep learning in a manner as profound as GPUs. However, existing software frameworks and training algorithms for deep learning have yet to evolve to fully leverage the capability of the new wave of silicon. We already see the limitations of existing algorithms for models that exploit structured input via complex and instance-dependent control flow, which prohibits minibatching. We present an asynchronous model-parallel (AMP) training algorithm that is specifically motivated by training on networks of interconnected devices. Through an implementation on multi-core CPUs, we show that AMP training converges to the same accuracy as conventional synchronous training algorithms in a similar number of epochs, but utilizes the available hardware more efficiently even for small minibatch sizes, resulting in significantly shorter overall training times. Our framework opens the door for scaling up a new class of deep learning models that cannot be efficiently trained today. |
| Tasks | |
| Published | 2017-05-27 |
| URL | http://arxiv.org/abs/1705.09786v3 |
| http://arxiv.org/pdf/1705.09786v3.pdf | |
| PWC | https://paperswithcode.com/paper/ampnet-asynchronous-model-parallel-training |
| Repo | |
| Framework | |
Integrating Lexical and Temporal Signals in Neural Ranking Models for Searching Social Media Streams
| Title | Integrating Lexical and Temporal Signals in Neural Ranking Models for Searching Social Media Streams |
| Authors | Jinfeng Rao, Hua He, Haotian Zhang, Ferhan Ture, Royal Sequiera, Salman Mohammed, Jimmy Lin |
| Abstract | Time is an important relevance signal when searching streams of social media posts. The distribution of document timestamps from the results of an initial query can be leveraged to infer the distribution of relevant documents, which can then be used to rerank the initial results. Previous experiments have shown that kernel density estimation is a simple yet effective implementation of this idea. This paper explores an alternative approach to mining temporal signals with recurrent neural networks. Our intuition is that neural networks provide a more expressive framework to capture the temporal coherence of neighboring documents in time. To our knowledge, we are the first to integrate lexical and temporal signals in an end-to-end neural network architecture, in which existing neural ranking models are used to generate query-document similarity vectors that feed into a bidirectional LSTM layer for temporal modeling. Our results are mixed: existing neural models for document ranking alone yield limited improvements over simple baselines, but the integration of lexical and temporal signals yield significant improvements over competitive temporal baselines. |
| Tasks | Density Estimation, Document Ranking |
| Published | 2017-07-25 |
| URL | http://arxiv.org/abs/1707.07792v1 |
| http://arxiv.org/pdf/1707.07792v1.pdf | |
| PWC | https://paperswithcode.com/paper/integrating-lexical-and-temporal-signals-in |
| Repo | |
| Framework | |
Word-Entity Duet Representations for Document Ranking
| Title | Word-Entity Duet Representations for Document Ranking |
| Authors | Chenyan Xiong, Jamie Callan, Tie-Yan Liu |
| Abstract | This paper presents a word-entity duet framework for utilizing knowledge bases in ad-hoc retrieval. In this work, the query and documents are modeled by word-based representations and entity-based representations. Ranking features are generated by the interactions between the two representations, incorporating information from the word space, the entity space, and the cross-space connections through the knowledge graph. To handle the uncertainties from the automatically constructed entity representations, an attention-based ranking model AttR-Duet is developed. With back-propagation from ranking labels, the model learns simultaneously how to demote noisy entities and how to rank documents with the word-entity duet. Evaluation results on TREC Web Track ad-hoc task demonstrate that all of the four-way interactions in the duet are useful, the attention mechanism successfully steers the model away from noisy entities, and together they significantly outperform both word-based and entity-based learning to rank systems. |
| Tasks | Document Ranking, Learning-To-Rank |
| Published | 2017-06-20 |
| URL | http://arxiv.org/abs/1706.06636v1 |
| http://arxiv.org/pdf/1706.06636v1.pdf | |
| PWC | https://paperswithcode.com/paper/word-entity-duet-representations-for-document |
| Repo | |
| Framework | |
On Type-Aware Entity Retrieval
| Title | On Type-Aware Entity Retrieval |
| Authors | Darío Garigliotti, Krisztian Balog |
| Abstract | Today, the practice of returning entities from a knowledge base in response to search queries has become widespread. One of the distinctive characteristics of entities is that they are typed, i.e., assigned to some hierarchically organized type system (type taxonomy). The primary objective of this paper is to gain a better understanding of how entity type information can be utilized in entity retrieval. We perform this investigation in an idealized “oracle” setting, assuming that we know the distribution of target types of the relevant entities for a given query. We perform a thorough analysis of three main aspects: (i) the choice of type taxonomy, (ii) the representation of hierarchical type information, and (iii) the combination of type-based and term-based similarity in the retrieval model. Using a standard entity search test collection based on DBpedia, we find that type information proves most useful when using large type taxonomies that provide very specific types. We provide further insights on the extensional coverage of entities and on the utility of target types. |
| Tasks | |
| Published | 2017-08-28 |
| URL | http://arxiv.org/abs/1708.08291v1 |
| http://arxiv.org/pdf/1708.08291v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-type-aware-entity-retrieval |
| Repo | |
| Framework | |
An Interval-Based Bayesian Generative Model for Human Complex Activity Recognition
| Title | An Interval-Based Bayesian Generative Model for Human Complex Activity Recognition |
| Authors | Li Liu, Yongzhong Yang, Lakshmi Narasimhan Govindarajan, Shu Wang, Bin Hu, Li Cheng, David S. Rosenblum |
| Abstract | Complex activity recognition is challenging due to the inherent uncertainty and diversity of performing a complex activity. Normally, each instance of a complex activity has its own configuration of atomic actions and their temporal dependencies. We propose in this paper an atomic action-based Bayesian model that constructs Allen’s interval relation networks to characterize complex activities with structural varieties in a probabilistic generative way: By introducing latent variables from the Chinese restaurant process, our approach is able to capture all possible styles of a particular complex activity as a unique set of distributions over atomic actions and relations. We also show that local temporal dependencies can be retained and are globally consistent in the resulting interval network. Moreover, network structure can be learned from empirical data. A new dataset of complex hand activities has been constructed and made publicly available, which is much larger in size than any existing datasets. Empirical evaluations on benchmark datasets as well as our in-house dataset demonstrate the competitiveness of our approach. |
| Tasks | Activity Recognition |
| Published | 2017-01-04 |
| URL | http://arxiv.org/abs/1701.00903v1 |
| http://arxiv.org/pdf/1701.00903v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-interval-based-bayesian-generative-model |
| Repo | |
| Framework | |
Database of Parliamentary Speeches in Ireland, 1919-2013
| Title | Database of Parliamentary Speeches in Ireland, 1919-2013 |
| Authors | Alexander Herzog, Slava J. Mikhaylov |
| Abstract | We present a database of parliamentary debates that contains the complete record of parliamentary speeches from D'ail 'Eireann, the lower house and principal chamber of the Irish parliament, from 1919 to 2013. In addition, the database contains background information on all TDs (Teachta D'ala, members of parliament), such as their party affiliations, constituencies and office positions. The current version of the database includes close to 4.5 million speeches from 1,178 TDs. The speeches were downloaded from the official parliament website and further processed and parsed with a Python script. Background information on TDs was collected from the member database of the parliament website. Data on cabinet positions (ministers and junior ministers) was collected from the official website of the government. A record linkage algorithm and human coders were used to match TDs and ministers. |
| Tasks | |
| Published | 2017-08-15 |
| URL | http://arxiv.org/abs/1708.04557v1 |
| http://arxiv.org/pdf/1708.04557v1.pdf | |
| PWC | https://paperswithcode.com/paper/database-of-parliamentary-speeches-in-ireland |
| Repo | |
| Framework | |
Arc-swift: A Novel Transition System for Dependency Parsing
| Title | Arc-swift: A Novel Transition System for Dependency Parsing |
| Authors | Peng Qi, Christopher D. Manning |
| Abstract | Transition-based dependency parsers often need sequences of local shift and reduce operations to produce certain attachments. Correct individual decisions hence require global information about the sentence context and mistakes cause error propagation. This paper proposes a novel transition system, arc-swift, that enables direct attachments between tokens farther apart with a single transition. This allows the parser to leverage lexical information more directly in transition decisions. Hence, arc-swift can achieve significantly better performance with a very small beam size. Our parsers reduce error by 3.7–7.6% relative to those using existing transition systems on the Penn Treebank dependency parsing task and English Universal Dependencies. |
| Tasks | Dependency Parsing |
| Published | 2017-05-12 |
| URL | http://arxiv.org/abs/1705.04434v1 |
| http://arxiv.org/pdf/1705.04434v1.pdf | |
| PWC | https://paperswithcode.com/paper/arc-swift-a-novel-transition-system-for |
| Repo | |
| Framework | |
Learning When to Attend for Neural Machine Translation
| Title | Learning When to Attend for Neural Machine Translation |
| Authors | Junhui Li, Muhua Zhu |
| Abstract | In the past few years, attention mechanisms have become an indispensable component of end-to-end neural machine translation models. However, previous attention models always refer to some source words when predicting a target word, which contradicts with the fact that some target words have no corresponding source words. Motivated by this observation, we propose a novel attention model that has the capability of determining when a decoder should attend to source words and when it should not. Experimental results on NIST Chinese-English translation tasks show that the new model achieves an improvement of 0.8 BLEU score over a state-of-the-art baseline. |
| Tasks | Machine Translation |
| Published | 2017-05-31 |
| URL | http://arxiv.org/abs/1705.11160v1 |
| http://arxiv.org/pdf/1705.11160v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-when-to-attend-for-neural-machine |
| Repo | |
| Framework | |
