May 6, 2019

2917 words 14 mins read

Paper Group ANR 391

Functional Hashing for Compressing Neural Networks. Robust and scalable Bayesian analysis of spatial neural tuning function data. Optimizing Spectral Learning for Parsing. Low-Rank Matrices on Graphs: Generalized Recovery & Applications. Sequence Training and Adaptation of Highway Deep Neural Networks. Lexical Based Semantic Orientation of Online C …

Functional Hashing for Compressing Neural Networks


Title	Functional Hashing for Compressing Neural Networks
Authors	Lei Shi, Shikun Feng, ZhifanZhu
Abstract	As the complexity of deep neural networks (DNNs) trend to grow to absorb the increasing sizes of data, memory and energy consumption has been receiving more and more attentions for industrial applications, especially on mobile devices. This paper presents a novel structure based on functional hashing to compress DNNs, namely FunHashNN. For each entry in a deep net, FunHashNN uses multiple low-cost hash functions to fetch values in the compression space, and then employs a small reconstruction network to recover that entry. The reconstruction network is plugged into the whole network and trained jointly. FunHashNN includes the recently proposed HashedNets as a degenerated case, and benefits from larger value capacity and less reconstruction loss. We further discuss extensions with dual space hashing and multi-hops. On several benchmark datasets, FunHashNN demonstrates high compression ratios with little loss on prediction accuracy.
Tasks
Published	2016-05-20
URL	http://arxiv.org/abs/1605.06560v1
PDF	http://arxiv.org/pdf/1605.06560v1.pdf
PWC	https://paperswithcode.com/paper/functional-hashing-for-compressing-neural
Repo
Framework

Robust and scalable Bayesian analysis of spatial neural tuning function data


Title	Robust and scalable Bayesian analysis of spatial neural tuning function data
Authors	Kamiar Rahnama Rad, Timothy A. Machado, Liam Paninski
Abstract	A common analytical problem in neuroscience is the interpretation of neural activity with respect to sensory input or behavioral output. This is typically achieved by regressing measured neural activity against known stimuli or behavioral variables to produce a “tuning function” for each neuron. Unfortunately, because this approach handles neurons individually, it cannot take advantage of simultaneous measurements from spatially adjacent neurons that often have similar tuning properties. On the other hand, sharing information between adjacent neurons can errantly degrade estimates of tuning functions across space if there are sharp discontinuities in tuning between nearby neurons. In this paper, we develop a computationally efficient block Gibbs sampler that effectively pools information between neurons to de-noise tuning function estimates while simultaneously preserving sharp discontinuities that might exist in the organization of tuning across space. This method is fully Bayesian and its computational cost per iteration scales sub-quadratically with total parameter dimensionality. We demonstrate the robustness and scalability of this approach by applying it to both real and synthetic datasets. In particular, an application to data from the spinal cord illustrates that the proposed methods can dramatically decrease the experimental time required to accurately estimate tuning functions.
Tasks
Published	2016-06-24
URL	http://arxiv.org/abs/1606.07845v1
PDF	http://arxiv.org/pdf/1606.07845v1.pdf
PWC	https://paperswithcode.com/paper/robust-and-scalable-bayesian-analysis-of
Repo
Framework

Optimizing Spectral Learning for Parsing


Title	Optimizing Spectral Learning for Parsing
Authors	Shashi Narayan, Shay B. Cohen
Abstract	We describe a search algorithm for optimizing the number of latent states when estimating latent-variable PCFGs with spectral methods. Our results show that contrary to the common belief that the number of latent states for each nonterminal in an L-PCFG can be decided in isolation with spectral methods, parsing results significantly improve if the number of latent states for each nonterminal is globally optimized, while taking into account interactions between the different nonterminals. In addition, we contribute an empirical analysis of spectral algorithms on eight morphologically rich languages: Basque, French, German, Hebrew, Hungarian, Korean, Polish and Swedish. Our results show that our estimation consistently performs better or close to coarse-to-fine expectation-maximization techniques for these languages.
Tasks
Published	2016-06-07
URL	http://arxiv.org/abs/1606.02342v3
PDF	http://arxiv.org/pdf/1606.02342v3.pdf
PWC	https://paperswithcode.com/paper/optimizing-spectral-learning-for-parsing
Repo
Framework

Low-Rank Matrices on Graphs: Generalized Recovery & Applications


Title	Low-Rank Matrices on Graphs: Generalized Recovery & Applications
Authors	Nauman Shahid, Nathanael Perraudin, Pierre Vandergheynst
Abstract	Many real world datasets subsume a linear or non-linear low-rank structure in a very low-dimensional space. Unfortunately, one often has very little or no information about the geometry of the space, resulting in a highly under-determined recovery problem. Under certain circumstances, state-of-the-art algorithms provide an exact recovery for linear low-rank structures but at the expense of highly inscalable algorithms which use nuclear norm. However, the case of non-linear structures remains unresolved. We revisit the problem of low-rank recovery from a totally different perspective, involving graphs which encode pairwise similarity between the data samples and features. Surprisingly, our analysis confirms that it is possible to recover many approximate linear and non-linear low-rank structures with recovery guarantees with a set of highly scalable and efficient algorithms. We call such data matrices as \textit{Low-Rank matrices on graphs} and show that many real world datasets satisfy this assumption approximately due to underlying stationarity. Our detailed theoretical and experimental analysis unveils the power of the simple, yet very novel recovery framework \textit{Fast Robust PCA on Graphs}
Tasks
Published	2016-05-18
URL	http://arxiv.org/abs/1605.05579v3
PDF	http://arxiv.org/pdf/1605.05579v3.pdf
PWC	https://paperswithcode.com/paper/low-rank-matrices-on-graphs-generalized
Repo
Framework

Sequence Training and Adaptation of Highway Deep Neural Networks


Title	Sequence Training and Adaptation of Highway Deep Neural Networks
Authors	Liang Lu
Abstract	Highway deep neural network (HDNN) is a type of depth-gated feedforward neural network, which has shown to be easier to train with more hidden layers and also generalise better compared to conventional plain deep neural networks (DNNs). Previously, we investigated a structured HDNN architecture for speech recognition, in which the two gate functions were tied across all the hidden layers, and we were able to train a much smaller model without sacrificing the recognition accuracy. In this paper, we carry on the study of this architecture with sequence-discriminative training criterion and speaker adaptation techniques on the AMI meeting speech recognition corpus. We show that these two techniques improve speech recognition accuracy on top of the model trained with the cross entropy criterion. Furthermore, we demonstrate that the two gate functions that are tied across all the hidden layers are able to control the information flow over the whole network, and we can achieve considerable improvements by only updating these gate functions in both sequence training and adaptation experiments.
Tasks	Speech Recognition
Published	2016-07-07
URL	http://arxiv.org/abs/1607.01963v5
PDF	http://arxiv.org/pdf/1607.01963v5.pdf
PWC	https://paperswithcode.com/paper/sequence-training-and-adaptation-of-highway
Repo
Framework

Lexical Based Semantic Orientation of Online Customer Reviews and Blogs


Title	Lexical Based Semantic Orientation of Online Customer Reviews and Blogs
Authors	Aurangzeb khan, Khairullah khan, Shakeel Ahmad, Fazal Masood Kundi, Irum Tareen, Muhammad Zubair Asghar
Abstract	Rapid increase in internet users along with growing power of online review sites and social media has given birth to sentiment analysis or opinion mining, which aims at determining what other people think and comment. Sentiments or Opinions contain public generated content about products, services, policies and politics. People are usually interested to seek positive and negative opinions containing likes and dislikes, shared by users for features of particular product or service. This paper proposed sentence-level lexical based domain independent sentiment classification method for different types of data such as reviews and blogs. The proposed method is based on general lexicons i.e. WordNet, SentiWordNet and user defined lexical dictionaries for semantic orientation. The relations and glosses of these dictionaries provide solution to the domain portability problem. The method performs better than word and text level corpus based machine learning methods for semantic orientation. The results show the proposed method performs better as it shows precision of 87% and83% at document and sentence levels respectively for online comments.
Tasks	Opinion Mining, Sentiment Analysis
Published	2016-07-08
URL	http://arxiv.org/abs/1607.02355v1
PDF	http://arxiv.org/pdf/1607.02355v1.pdf
PWC	https://paperswithcode.com/paper/lexical-based-semantic-orientation-of-online
Repo
Framework

Camera Fingerprint: A New Perspective for Identifying User’s Identity


Title	Camera Fingerprint: A New Perspective for Identifying User’s Identity
Authors	Xiang Jiang, Shikui Wei, Ruizhen Zhao, Yao Zhao, Xindong Wu
Abstract	Identifying user’s identity is a key problem in many data mining applications, such as product recommendation, customized content delivery and criminal identification. Given a set of accounts from the same or different social network platforms, user identification attempts to identify all accounts belonging to the same person. A commonly used solution is to build the relationship among different accounts by exploring their collective patterns, e.g., user profile, writing style, similar comments. However, this kind of method doesn’t work well in many practical scenarios, since the information posted explicitly by users may be false due to various reasons. In this paper, we re-inspect the user identification problem from a novel perspective, i.e., identifying user’s identity by matching his/her cameras. The underlying assumption is that multiple accounts belonging to the same person contain the same or similar camera fingerprint information. The proposed framework, called User Camera Identification (UCI), is based on camera fingerprints, which takes fully into account the problems of multiple cameras and reposting behaviors.
Tasks	Product Recommendation
Published	2016-10-25
URL	http://arxiv.org/abs/1610.07728v1
PDF	http://arxiv.org/pdf/1610.07728v1.pdf
PWC	https://paperswithcode.com/paper/camera-fingerprint-a-new-perspective-for
Repo
Framework

The Bayesian Low-Rank Determinantal Point Process Mixture Model


Title	The Bayesian Low-Rank Determinantal Point Process Mixture Model
Authors	Mike Gartrell, Ulrich Paquet, Noam Koenigstein
Abstract	Determinantal point processes (DPPs) are an elegant model for encoding probabilities over subsets, such as shopping baskets, of a ground set, such as an item catalog. They are useful for a number of machine learning tasks, including product recommendation. DPPs are parametrized by a positive semi-definite kernel matrix. Recent work has shown that using a low-rank factorization of this kernel provides remarkable scalability improvements that open the door to training on large-scale datasets and computing online recommendations, both of which are infeasible with standard DPP models that use a full-rank kernel. In this paper we present a low-rank DPP mixture model that allows us to represent the latent structure present in observed subsets as a mixture of a number of component low-rank DPPs, where each component DPP is responsible for representing a portion of the observed data. The mixture model allows us to effectively address the capacity constraints of the low-rank DPP model. We present an efficient and scalable Markov Chain Monte Carlo (MCMC) learning algorithm for our model that uses Gibbs sampling and stochastic gradient Hamiltonian Monte Carlo (SGHMC). Using an evaluation on several real-world product recommendation datasets, we show that our low-rank DPP mixture model provides substantially better predictive performance than is possible with a single low-rank or full-rank DPP, and significantly better performance than several other competing recommendation methods in many cases.
Tasks	Point Processes, Product Recommendation
Published	2016-08-15
URL	http://arxiv.org/abs/1608.04245v2
PDF	http://arxiv.org/pdf/1608.04245v2.pdf
PWC	https://paperswithcode.com/paper/the-bayesian-low-rank-determinantal-point
Repo
Framework

Evolutionary-aided negotiation model for bilateral bargaining in Ambient Intelligence domains with complex utility functions


Title	Evolutionary-aided negotiation model for bilateral bargaining in Ambient Intelligence domains with complex utility functions
Authors	Victor Sanchez-Anguix, Soledad Valero, Vicente Julian, Vicente Botti, Ana Garcia-Fornes
Abstract	Ambient Intelligence aims to offer personalized services and easier ways of interaction between people and systems. Since several users and systems may coexist in these environments, it is quite possible that entities with opposing preferences need to cooperate to reach their respective goals. Automated negotiation is pointed as one of the mechanisms that may provide a solution to this kind of problems. In this article, a multi-issue bilateral bargaining model for Ambient Intelligence domains is presented where it is assumed that agents have computational bounded resources and do not know their opponents’ preferences. The main goal of this work is to provide negotiation models that obtain efficient agreements while maintaining the computational cost low. A niching genetic algorithm is used before the negotiation process to sample one’s own utility function (self-sampling). During the negotiation process, genetic operators are applied over the opponent’s and one’s own offers in order to sample new offers that are interesting for both parties. Results show that the proposed model is capable of outperforming similarity heuristics which only sample before the negotiation process and of obtaining similar results to similarity heuristics which have access to all of the possible offers.
Tasks
Published	2016-04-16
URL	http://arxiv.org/abs/1604.04730v1
PDF	http://arxiv.org/pdf/1604.04730v1.pdf
PWC	https://paperswithcode.com/paper/evolutionary-aided-negotiation-model-for
Repo
Framework

Generalization Error of Invariant Classifiers


Title	Generalization Error of Invariant Classifiers
Authors	Jure Sokolic, Raja Giryes, Guillermo Sapiro, Miguel R. D. Rodrigues
Abstract	This paper studies the generalization error of invariant classifiers. In particular, we consider the common scenario where the classification task is invariant to certain transformations of the input, and that the classifier is constructed (or learned) to be invariant to these transformations. Our approach relies on factoring the input space into a product of a base space and a set of transformations. We show that whereas the generalization error of a non-invariant classifier is proportional to the complexity of the input space, the generalization error of an invariant classifier is proportional to the complexity of the base space. We also derive a set of sufficient conditions on the geometry of the base space and the set of transformations that ensure that the complexity of the base space is much smaller than the complexity of the input space. Our analysis applies to general classifiers such as convolutional neural networks. We demonstrate the implications of the developed theory for such classifiers with experiments on the MNIST and CIFAR-10 datasets.
Tasks
Published	2016-10-14
URL	http://arxiv.org/abs/1610.04574v3
PDF	http://arxiv.org/pdf/1610.04574v3.pdf
PWC	https://paperswithcode.com/paper/generalization-error-of-invariant-classifiers
Repo
Framework

Strong Neutrosophic Graphs and Subgraph Topological Subspaces


Title	Strong Neutrosophic Graphs and Subgraph Topological Subspaces
Authors	W. B. Vasantha Kandasamy, Ilanthenral K, Florentin Smarandache
Abstract	In this book authors for the first time introduce the notion of strong neutrosophic graphs. They are very different from the usual graphs and neutrosophic graphs. Using these new structures special subgraph topological spaces are defined. Further special lattice graph of subgraphs of these graphs are defined and described. Several interesting properties using subgraphs of a strong neutrosophic graph are obtained. Several open conjectures are proposed. These new class of strong neutrosophic graphs will certainly find applications in Neutrosophic Cognitive Maps (NCM), Neutrosophic Relational Maps (NRM) and Neutrosophic Relational Equations (NRE) with appropriate modifications.
Tasks
Published	2016-10-30
URL	http://arxiv.org/abs/1611.00576v1
PDF	http://arxiv.org/pdf/1611.00576v1.pdf
PWC	https://paperswithcode.com/paper/strong-neutrosophic-graphs-and-subgraph
Repo
Framework

Scalable Discrete Supervised Hash Learning with Asymmetric Matrix Factorization


Title	Scalable Discrete Supervised Hash Learning with Asymmetric Matrix Factorization
Authors	Shifeng Zhang, Jianmin Li, Jinma Guo, Bo Zhang
Abstract	Hashing method maps similar data to binary hashcodes with smaller hamming distance, and it has received a broad attention due to its low storage cost and fast retrieval speed. However, the existing limitations make the present algorithms difficult to deal with large-scale datasets: (1) discrete constraints are involved in the learning of the hash function; (2) pairwise or triplet similarity is adopted to generate efficient hashcodes, resulting both time and space complexity are greater than O(n^2). To address these issues, we propose a novel discrete supervised hash learning framework which can be scalable to large-scale datasets. First, the discrete learning procedure is decomposed into a binary classifier learning scheme and binary codes learning scheme, which makes the learning procedure more efficient. Second, we adopt the Asymmetric Low-rank Matrix Factorization and propose the Fast Clustering-based Batch Coordinate Descent method, such that the time and space complexity is reduced to O(n). The proposed framework also provides a flexible paradigm to incorporate with arbitrary hash function, including deep neural networks and kernel methods. Experiments on large-scale datasets demonstrate that the proposed method is superior or comparable with state-of-the-art hashing algorithms.
Tasks
Published	2016-09-28
URL	http://arxiv.org/abs/1609.08740v1
PDF	http://arxiv.org/pdf/1609.08740v1.pdf
PWC	https://paperswithcode.com/paper/scalable-discrete-supervised-hash-learning
Repo
Framework

Direct Method for Training Feed-forward Neural Networks using Batch Extended Kalman Filter for Multi-Step-Ahead Predictions


Title	Direct Method for Training Feed-forward Neural Networks using Batch Extended Kalman Filter for Multi-Step-Ahead Predictions
Authors	Artem Chernodub
Abstract	This paper is dedicated to the long-term, or multi-step-ahead, time series prediction problem. We propose a novel method for training feed-forward neural networks, such as multilayer perceptrons, with tapped delay lines. Special batch calculation of derivatives called Forecasted Propagation Through Time and batch modification of the Extended Kalman Filter are introduced. Experiments were carried out on well-known time series benchmarks, the Mackey-Glass chaotic process and the Santa Fe Laser Data Series. Recurrent and feed-forward neural networks were evaluated.
Tasks	Time Series, Time Series Prediction
Published	2016-05-12
URL	http://arxiv.org/abs/1605.03764v1
PDF	http://arxiv.org/pdf/1605.03764v1.pdf
PWC	https://paperswithcode.com/paper/direct-method-for-training-feed-forward
Repo
Framework

An equation-of-state-meter of QCD transition from deep learning


Title	An equation-of-state-meter of QCD transition from deep learning
Authors	Long-Gang Pang, Kai Zhou, Nan Su, Hannah Petersen, Horst Stöcker, Xin-Nian Wang
Abstract	Supervised learning with a deep convolutional neural network is used to identify the QCD equation of state (EoS) employed in relativistic hydrodynamic simulations of heavy-ion collisions from the simulated final-state particle spectra $\rho(p_T,\Phi)$. High-level correlations of $\rho(p_T,\Phi)$ learned by the neural network act as an effective “EoS-meter” in detecting the nature of the QCD transition. The EoS-meter is model independent and insensitive to other simulation inputs, especially the initial conditions. Thus it provides a powerful direct-connection of heavy-ion collision observables with the bulk properties of QCD.
Tasks
Published	2016-12-13
URL	http://arxiv.org/abs/1612.04262v3
PDF	http://arxiv.org/pdf/1612.04262v3.pdf
PWC	https://paperswithcode.com/paper/an-equation-of-state-meter-of-qcd-transition
Repo
Framework

Horizontally Scalable Submodular Maximization


Title	Horizontally Scalable Submodular Maximization
Authors	Mario Lucic, Olivier Bachem, Morteza Zadimoghaddam, Andreas Krause
Abstract	A variety of large-scale machine learning problems can be cast as instances of constrained submodular maximization. Existing approaches for distributed submodular maximization have a critical drawback: The capacity - number of instances that can fit in memory - must grow with the data set size. In practice, while one can provision many machines, the capacity of each machine is limited by physical constraints. We propose a truly scalable approach for distributed submodular maximization under fixed capacity. The proposed framework applies to a broad class of algorithms and constraints and provides theoretical guarantees on the approximation factor for any available capacity. We empirically evaluate the proposed algorithm on a variety of data sets and demonstrate that it achieves performance competitive with the centralized greedy solution.
Tasks
Published	2016-05-31
URL	http://arxiv.org/abs/1605.09619v1
PDF	http://arxiv.org/pdf/1605.09619v1.pdf
PWC	https://paperswithcode.com/paper/horizontally-scalable-submodular-maximization
Repo
Framework