Paper Group ANR 372
Neural Contextual Conversation Learning with Labeled Question-Answering Pairs. Learning by Stimulation Avoidance: A Principle to Control Spiking Neural Networks Dynamics. Adding Context to Concept Trees. Template shape estimation: correcting an asymptotic bias. Efficient Character-level Document Classification by Combining Convolution and Recurrent …
Neural Contextual Conversation Learning with Labeled Question-Answering Pairs
| Title | Neural Contextual Conversation Learning with Labeled Question-Answering Pairs |
| Authors | Kun Xiong, Anqi Cui, Zefeng Zhang, Ming Li |
| Abstract | Neural conversational models tend to produce generic or safe responses in different contexts, e.g., reply \textit{“Of course”} to narrative statements or \textit{“I don’t know”} to questions. In this paper, we propose an end-to-end approach to avoid such problem in neural generative models. Additional memory mechanisms have been introduced to standard sequence-to-sequence (seq2seq) models, so that context can be considered while generating sentences. Three seq2seq models, which memorize a fix-sized contextual vector from hidden input, hidden input/output and a gated contextual attention structure respectively, have been trained and tested on a dataset of labeled question-answering pairs in Chinese. The model with contextual attention outperforms others including the state-of-the-art seq2seq models on perplexity test. The novel contextual model generates diverse and robust responses, and is able to carry out conversations on a wide range of topics appropriately. |
| Tasks | Question Answering |
| Published | 2016-07-20 |
| URL | http://arxiv.org/abs/1607.05809v1 |
| http://arxiv.org/pdf/1607.05809v1.pdf | |
| PWC | https://paperswithcode.com/paper/neural-contextual-conversation-learning-with |
| Repo | |
| Framework | |
Learning by Stimulation Avoidance: A Principle to Control Spiking Neural Networks Dynamics
| Title | Learning by Stimulation Avoidance: A Principle to Control Spiking Neural Networks Dynamics |
| Authors | Lana Sinapayen, Atsushi Masumori, Takashi Ikegami |
| Abstract | Learning based on networks of real neurons, and by extension biologically inspired models of neural networks, has yet to find general learning rules leading to widespread applications. In this paper, we argue for the existence of a principle allowing to steer the dynamics of a biologically inspired neural network. Using carefully timed external stimulation, the network can be driven towards a desired dynamical state. We term this principle “Learning by Stimulation Avoidance” (LSA). We demonstrate through simulation that the minimal sufficient conditions leading to LSA in artificial networks are also sufficient to reproduce learning results similar to those obtained in biological neurons by Shahaf and Marom [1]. We examine the mechanism’s basic dynamics in a reduced network, and demonstrate how it scales up to a network of 100 neurons. We show that LSA has a higher explanatory power than existing hypotheses about the response of biological neural networks to external simulation, and can be used as a learning rule for an embodied application: learning of wall avoidance by a simulated robot. The surge in popularity of artificial neural networks is mostly directed to disembodied models of neurons with biologically irrelevant dynamics: to the authors’ knowledge, this is the first work demonstrating sensory-motor learning with random spiking networks through pure Hebbian learning. |
| Tasks | |
| Published | 2016-09-25 |
| URL | http://arxiv.org/abs/1609.07706v2 |
| http://arxiv.org/pdf/1609.07706v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-by-stimulation-avoidance-a-principle |
| Repo | |
| Framework | |
Adding Context to Concept Trees
| Title | Adding Context to Concept Trees |
| Authors | Kieran Greer |
| Abstract | A Concept Tree is a structure for storing knowledge where the trees are stored in a database called a Concept Base. It sits between the highly distributed neural architectures and the distributed information systems, with the intention of bringing brain-like and computer systems closer together. Concept Trees can grow from the semi-structured sources when consistent sequences of concepts are presented. Each tree ideally represents a single cohesive concept and the trees can link with each other for navigation and semantic purposes. The trees are therefore also a type of semantic network and would benefit from having a consistent level of context for each node. A consistent build process is managed through a ‘counting rule’ and some other rules that can normalise the database structure. This restricted structure can then be complimented and enriched by the more dynamic context. It is also suggested to use the linking structure of the licas system [15] as a basis for the context links, where the mathematical model is extended further to define this. A number of tests have demonstrated the soundness of the architecture. Building the trees from text documents shows that the tree structure could be inherent in natural language. Then, two types of query language are described. Both of these can perform consistent query processes to return knowledge to the user and even enhance the query with new knowledge. This is supported even further with direct comparisons to a cognitive model, also being developed by the author. |
| Tasks | |
| Published | 2016-06-17 |
| URL | https://arxiv.org/abs/1606.05597v5 |
| https://arxiv.org/pdf/1606.05597v5.pdf | |
| PWC | https://paperswithcode.com/paper/adding-context-to-concept-trees |
| Repo | |
| Framework | |
Template shape estimation: correcting an asymptotic bias
| Title | Template shape estimation: correcting an asymptotic bias |
| Authors | Nina Miolane, Susan Holmes, Xavier Pennec |
| Abstract | We use tools from geometric statistics to analyze the usual estimation procedure of a template shape. This applies to shapes from landmarks, curves, surfaces, images etc. We demonstrate the asymptotic bias of the template shape estimation using the stratified geometry of the shape space. We give a Taylor expansion of the bias with respect to a parameter $\sigma$ describing the measurement error on the data. We propose two bootstrap procedures that quantify the bias and correct it, if needed. They are applicable for any type of shape data. We give a rule of thumb to provide intuition on whether the bias has to be corrected. This exhibits the parameters that control the bias’ magnitude. We illustrate our results on simulated and real shape data. |
| Tasks | |
| Published | 2016-09-06 |
| URL | http://arxiv.org/abs/1610.01502v2 |
| http://arxiv.org/pdf/1610.01502v2.pdf | |
| PWC | https://paperswithcode.com/paper/template-shape-estimation-correcting-an |
| Repo | |
| Framework | |
Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers
| Title | Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers |
| Authors | Yijun Xiao, Kyunghyun Cho |
| Abstract | Document classification tasks were primarily tackled at word level. Recent research that works with character-level inputs shows several benefits over word-level approaches such as natural incorporation of morphemes and better handling of rare words. We propose a neural network architecture that utilizes both convolution and recurrent layers to efficiently encode character inputs. We validate the proposed model on eight large scale document classification tasks and compare with character-level convolution-only models. It achieves comparable performances with much less parameters. |
| Tasks | Document Classification |
| Published | 2016-02-01 |
| URL | http://arxiv.org/abs/1602.00367v1 |
| http://arxiv.org/pdf/1602.00367v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-character-level-document |
| Repo | |
| Framework | |
Deep Self-Convolutional Activations Descriptor for Dense Cross-Modal Correspondence
| Title | Deep Self-Convolutional Activations Descriptor for Dense Cross-Modal Correspondence |
| Authors | Seungryong Kim, Dongbo Min, Stephen Lin, Kwanghoon Sohn |
| Abstract | We present a novel descriptor, called deep self-convolutional activations (DeSCA), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions. Motivated by descriptors based on local self-similarity (LSS), we formulate a novel descriptor by leveraging LSS in a deep architecture, leading to better discriminative power and greater robustness to non-rigid image deformations than state-of-the-art cross-modality descriptors. The DeSCA first computes self-convolutions over a local support window for randomly sampled patches, and then builds self-convolution activations by performing an average pooling through a hierarchical formulation within a deep convolutional architecture. Finally, the feature responses on the self-convolution activations are encoded through a spatial pyramid pooling in a circular configuration. In contrast to existing convolutional neural networks (CNNs) based descriptors, the DeSCA is training-free (i.e., randomly sampled patches are utilized as the convolution kernels), is robust to cross-modal imaging, and can be densely computed in an efficient manner that significantly reduces computational redundancy. The state-of-the-art performance of DeSCA on challenging cases of cross-modal image pairs is demonstrated through extensive experiments. |
| Tasks | |
| Published | 2016-03-21 |
| URL | http://arxiv.org/abs/1603.06327v1 |
| http://arxiv.org/pdf/1603.06327v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-self-convolutional-activations |
| Repo | |
| Framework | |
Weakly-supervised Learning of Mid-level Features for Pedestrian Attribute Recognition and Localization
| Title | Weakly-supervised Learning of Mid-level Features for Pedestrian Attribute Recognition and Localization |
| Authors | Kai Yu, Biao Leng, Zhang Zhang, Dangwei Li, Kaiqi Huang |
| Abstract | State-of-the-art methods treat pedestrian attribute recognition as a multi-label image classification problem. The location information of person attributes is usually eliminated or simply encoded in the rigid splitting of whole body in previous work. In this paper, we formulate the task in a weakly-supervised attribute localization framework. Based on GoogLeNet, firstly, a set of mid-level attribute features are discovered by novelly designed detection layers, where a max-pooling based weakly-supervised object detection technique is used to train these layers with only image-level labels without the need of bounding box annotations of pedestrian attributes. Secondly, attribute labels are predicted by regression of the detection response magnitudes. Finally, the locations and rough shapes of pedestrian attributes can be inferred by performing clustering on a fusion of activation maps of the detection layers, where the fusion weights are estimated as the correlation strengths between each attribute and its relevant mid-level features. Extensive experiments are performed on the two currently largest pedestrian attribute datasets, i.e. the PETA dataset and the RAP dataset. Results show that the proposed method has achieved competitive performance on attribute recognition, compared to other state-of-the-art methods. Moreover, the results of attribute localization are visualized to understand the characteristics of the proposed method. |
| Tasks | Image Classification, Object Detection, Pedestrian Attribute Recognition, Weakly Supervised Object Detection |
| Published | 2016-11-17 |
| URL | http://arxiv.org/abs/1611.05603v1 |
| http://arxiv.org/pdf/1611.05603v1.pdf | |
| PWC | https://paperswithcode.com/paper/weakly-supervised-learning-of-mid-level |
| Repo | |
| Framework | |
Building Diversified Multiple Trees for Classification in High Dimensional Noisy Biomedical Data
| Title | Building Diversified Multiple Trees for Classification in High Dimensional Noisy Biomedical Data |
| Authors | Jiuyong Li, Lin Liu, Jixue Liu, Ryan Green |
| Abstract | It is common that a trained classification model is applied to the operating data that is deviated from the training data because of noise. This paper demonstrates that an ensemble classifier, Diversified Multiple Tree (DMT), is more robust in classifying noisy data than other widely used ensemble methods. DMT is tested on three real world biomedical data sets from different laboratories in comparison with four benchmark ensemble classifiers. Experimental results show that DMT is significantly more accurate than other benchmark ensemble classifiers on noisy test data. We also discuss a limitation of DMT and its possible variations. |
| Tasks | |
| Published | 2016-12-18 |
| URL | http://arxiv.org/abs/1612.05888v2 |
| http://arxiv.org/pdf/1612.05888v2.pdf | |
| PWC | https://paperswithcode.com/paper/building-diversified-multiple-trees-for |
| Repo | |
| Framework | |
Private Learning on Networks
| Title | Private Learning on Networks |
| Authors | Shripad Gade, Nitin H. Vaidya |
| Abstract | Continual data collection and widespread deployment of machine learning algorithms, particularly the distributed variants, have raised new privacy challenges. In a distributed machine learning scenario, the dataset is stored among several machines and they solve a distributed optimization problem to collectively learn the underlying model. We present a secure multi-party computation inspired privacy preserving distributed algorithm for optimizing a convex function consisting of several possibly non-convex functions. Each individual objective function is privately stored with an agent while the agents communicate model parameters with neighbor machines connected in a network. We show that our algorithm can correctly optimize the overall objective function and learn the underlying model accurately. We further prove that under a vertex connectivity condition on the topology, our algorithm preserves privacy of individual objective functions. We establish limits on the what a coalition of adversaries can learn by observing the messages and states shared over a network. |
| Tasks | Distributed Optimization |
| Published | 2016-12-15 |
| URL | http://arxiv.org/abs/1612.05236v1 |
| http://arxiv.org/pdf/1612.05236v1.pdf | |
| PWC | https://paperswithcode.com/paper/private-learning-on-networks |
| Repo | |
| Framework | |
Arbitrage-Free Combinatorial Market Making via Integer Programming
| Title | Arbitrage-Free Combinatorial Market Making via Integer Programming |
| Authors | Christian Kroer, Miroslav Dudík, Sébastien Lahaie, Sivaraman Balakrishnan |
| Abstract | We present a new combinatorial market maker that operates arbitrage-free combinatorial prediction markets specified by integer programs. Although the problem of arbitrage-free pricing, while maintaining a bound on the subsidy provided by the market maker, is #P-hard in the worst case, we posit that the typical case might be amenable to modern integer programming (IP) solvers. At the crux of our method is the Frank-Wolfe (conditional gradient) algorithm which is used to implement a Bregman projection aligned with the market maker’s cost function, using an IP solver as an oracle. We demonstrate the tractability and improved accuracy of our approach on real-world prediction market data from combinatorial bets placed on the 2010 NCAA Men’s Division I Basketball Tournament, where the outcome space is of size 2^63. To our knowledge, this is the first implementation and empirical evaluation of an arbitrage-free combinatorial prediction market on this scale. |
| Tasks | |
| Published | 2016-06-09 |
| URL | http://arxiv.org/abs/1606.02825v2 |
| http://arxiv.org/pdf/1606.02825v2.pdf | |
| PWC | https://paperswithcode.com/paper/arbitrage-free-combinatorial-market-making |
| Repo | |
| Framework | |
Consistency Analysis for the Doubly Stochastic Dirichlet Process
| Title | Consistency Analysis for the Doubly Stochastic Dirichlet Process |
| Authors | Xing Sun, Nelson H. C. Yung, Edmund Y. Lam, Hayden K. -H. So |
| Abstract | This technical report proves components consistency for the Doubly Stochastic Dirichlet Process with exponential convergence of posterior probability. We also present the fundamental properties for DSDP as well as inference algorithms. Simulation toy experiment and real-world experiment results for single and multi-cluster also support the consistency proof. This report is also a support document for the paper “Computationally Efficient Hyperspectral Data Learning Based on the Doubly Stochastic Dirichlet Process”. |
| Tasks | |
| Published | 2016-05-24 |
| URL | http://arxiv.org/abs/1605.07358v1 |
| http://arxiv.org/pdf/1605.07358v1.pdf | |
| PWC | https://paperswithcode.com/paper/consistency-analysis-for-the-doubly |
| Repo | |
| Framework | |
GeoTextTagger: High-Precision Location Tagging of Textual Documents using a Natural Language Processing Approach
| Title | GeoTextTagger: High-Precision Location Tagging of Textual Documents using a Natural Language Processing Approach |
| Authors | Shawn Brunsting, Hans De Sterck, Remco Dolman, Teun van Sprundel |
| Abstract | Location tagging, also known as geotagging or geolocation, is the process of assigning geographical coordinates to input data. In this paper we present an algorithm for location tagging of textual documents. Our approach makes use of previous work in natural language processing by using a state-of-the-art part-of-speech tagger and named entity recognizer to find blocks of text which may refer to locations. A knowledge base (OpenStreatMap) is then used to find a list of possible locations for each block. Finally, one location is chosen for each block by assigning distance-based scores to each location and repeatedly selecting the location and block with the best score. We tested our geolocation algorithm with Wikipedia articles about topics with a well-defined geographical location that are geotagged by the articles’ authors, where classification approaches have achieved median errors as low as 11 km, with attainable accuracy limited by the class size. Our approach achieved a 10th percentile error of 490 metres and median error of 54 kilometres on the Wikipedia dataset we used. When considering the five location tags with the greatest scores, 50% of articles were assigned at least one tag within 8.5 kilometres of the article’s author-assigned true location. We also tested our approach on Twitter messages that are tagged with the location from which the message was sent. Twitter texts are challenging because they are short and unstructured and often do not contain words referring to the location they were sent from, but we obtain potentially useful results. We explain how we use the Spark framework for data analytics to collect and process our test data. In general, classification-based approaches for location tagging may be reaching their upper accuracy limit, but our precision-focused approach has high accuracy for some texts and shows significant potential for improvement overall. |
| Tasks | |
| Published | 2016-01-22 |
| URL | http://arxiv.org/abs/1601.05893v1 |
| http://arxiv.org/pdf/1601.05893v1.pdf | |
| PWC | https://paperswithcode.com/paper/geotexttagger-high-precision-location-tagging |
| Repo | |
| Framework | |
Learning with Hierarchical Gaussian Kernels
| Title | Learning with Hierarchical Gaussian Kernels |
| Authors | Ingo Steinwart, Philipp Thomann, Nico Schmid |
| Abstract | We investigate iterated compositions of weighted sums of Gaussian kernels and provide an interpretation of the construction that shows some similarities with the architectures of deep neural networks. On the theoretical side, we show that these kernels are universal and that SVMs using these kernels are universally consistent. We further describe a parameter optimization method for the kernel parameters and empirically compare this method to SVMs, random forests, a multiple kernel learning approach, and to some deep neural networks. |
| Tasks | |
| Published | 2016-12-02 |
| URL | http://arxiv.org/abs/1612.00824v1 |
| http://arxiv.org/pdf/1612.00824v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-with-hierarchical-gaussian-kernels |
| Repo | |
| Framework | |
Asymptotic consistency and order specification for logistic classifier chains in multi-label learning
| Title | Asymptotic consistency and order specification for logistic classifier chains in multi-label learning |
| Authors | Paweł Teisseyre |
| Abstract | Classifier chains are popular and effective method to tackle a multi-label classification problem. The aim of this paper is to study the asymptotic properties of the chain model in which the conditional probabilities are of the logistic form. In particular we find conditions on the number of labels and the distribution of feature vector under which the estimated mode of the joint distribution of labels converges to the true mode. Best of our knowledge, this important issue has not yet been studied in the context of multi-label learning. We also investigate how the order of model building in a chain influences the estimation of the joint distribution of labels. We establish the link between the problem of incorrect ordering in the chain and incorrect model specification. We propose a procedure of determining the optimal ordering of labels in the chain, which is based on using measures of correct specification and allows to find the ordering such that the consecutive logistic models are best possibly specified. The other important question raised in this paper is how accurately can we estimate the joint posterior probability when the ordering of labels is wrong or the logistic models in the chain are incorrectly specified. The numerical experiments illustrate the theoretical results. |
| Tasks | Multi-Label Classification, Multi-Label Learning |
| Published | 2016-02-24 |
| URL | http://arxiv.org/abs/1602.07466v1 |
| http://arxiv.org/pdf/1602.07466v1.pdf | |
| PWC | https://paperswithcode.com/paper/asymptotic-consistency-and-order |
| Repo | |
| Framework | |
A Perspective on Deep Imaging
| Title | A Perspective on Deep Imaging |
| Authors | Ge Wang |
| Abstract | The combination of tomographic imaging and deep learning, or machine learning in general, promises to empower not only image analysis but also image reconstruction. The latter aspect is considered in this perspective article with an emphasis on medical imaging to develop a new generation of image reconstruction theories and techniques. This direction might lead to intelligent utilization of domain knowledge from big data, innovative approaches for image reconstruction, and superior performance in clinical and preclinical applications. To realize the full impact of machine learning on medical imaging, major challenges must be addressed. |
| Tasks | Image Reconstruction |
| Published | 2016-09-10 |
| URL | http://arxiv.org/abs/1609.04375v2 |
| http://arxiv.org/pdf/1609.04375v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-perspective-on-deep-imaging |
| Repo | |
| Framework | |
