July 27, 2019

2984 words 15 mins read

Paper Group ANR 752

Communication-Optimal Distributed Clustering. Query expansion techniques for information retrieval: A survey. Goal-Driven Dynamics Learning via Bayesian Optimization. Deep Triphone Embedding Improves Phoneme Recognition. Ideological Sublations: Resolution of Dialectic in Population-based Optimization. Saliency-based Sequential Image Attention with …

Communication-Optimal Distributed Clustering


Title	Communication-Optimal Distributed Clustering
Authors	Jiecao Chen, He Sun, David P. Woodruff, Qin Zhang
Abstract	Clustering large datasets is a fundamental problem with a number of applications in machine learning. Data is often collected on different sites and clustering needs to be performed in a distributed manner with low communication. We would like the quality of the clustering in the distributed setting to match that in the centralized setting for which all the data resides on a single site. In this work, we study both graph and geometric clustering problems in two distributed models: (1) a point-to-point model, and (2) a model with a broadcast channel. We give protocols in both models which we show are nearly optimal by proving almost matching communication lower bounds. Our work highlights the surprising power of a broadcast channel for clustering problems; roughly speaking, to spectrally cluster $n$ points or $n$ vertices in a graph distributed across $s$ servers, for a worst-case partitioning the communication complexity in a point-to-point model is $n \cdot s$, while in the broadcast model it is $n + s$. A similar phenomenon holds for the geometric setting as well. We implement our algorithms and demonstrate this phenomenon on real life datasets, showing that our algorithms are also very efficient in practice.
Tasks
Published	2017-02-01
URL	http://arxiv.org/abs/1702.00196v1
PDF	http://arxiv.org/pdf/1702.00196v1.pdf
PWC	https://paperswithcode.com/paper/communication-optimal-distributed-clustering
Repo
Framework

Query expansion techniques for information retrieval: A survey


Title	Query expansion techniques for information retrieval: A survey
Authors	Hiteshwar Kumar Azad, Akshay Deepak
Abstract	With the ever increasing size of the web, relevant information extraction on the Internet with a query formed by a few keywords has become a big challenge. Query Expansion (QE) plays a crucial role in improving searches on the Internet. Here, the user’s initial query is reformulated by adding additional meaningful terms with similar significance. QE – as part of information retrieval (IR) – has long attracted researchers’ attention. It has become very influential in the field of personalized social document, question answering, cross-language IR, information filtering and multimedia IR. Research in QE has gained further prominence because of IR dedicated conferences such as TREC (Text Information Retrieval Conference) and CLEF (Conference and Labs of the Evaluation Forum). This paper surveys QE techniques in IR from 1960 to 2017 with respect to core techniques, data sources used, weighting and ranking methodologies, user participation and applications – bringing out similarities and differences.
Tasks	Information Retrieval, Question Answering
Published	2017-08-01
URL	https://arxiv.org/abs/1708.00247v2
PDF	https://arxiv.org/pdf/1708.00247v2.pdf
PWC	https://paperswithcode.com/paper/query-expansion-techniques-for-information
Repo
Framework

Goal-Driven Dynamics Learning via Bayesian Optimization


Title	Goal-Driven Dynamics Learning via Bayesian Optimization
Authors	Somil Bansal, Roberto Calandra, Ted Xiao, Sergey Levine, Claire J. Tomlin
Abstract	Real-world robots are becoming increasingly complex and commonly act in poorly understood environments where it is extremely challenging to model or learn their true dynamics. Therefore, it might be desirable to take a task-specific approach, wherein the focus is on explicitly learning the dynamics model which achieves the best control performance for the task at hand, rather than learning the true dynamics. In this work, we use Bayesian optimization in an active learning framework where a locally linear dynamics model is learned with the intent of maximizing the control performance, and used in conjunction with optimal control schemes to efficiently design a controller for a given task. This model is updated directly based on the performance observed in experiments on the physical system in an iterative manner until a desired performance is achieved. We demonstrate the efficacy of the proposed approach through simulations and real experiments on a quadrotor testbed.
Tasks	Active Learning
Published	2017-03-27
URL	http://arxiv.org/abs/1703.09260v2
PDF	http://arxiv.org/pdf/1703.09260v2.pdf
PWC	https://paperswithcode.com/paper/goal-driven-dynamics-learning-via-bayesian
Repo
Framework

Deep Triphone Embedding Improves Phoneme Recognition


Title	Deep Triphone Embedding Improves Phoneme Recognition
Authors	Mohit Yadav, Vivek Tyagi
Abstract	In this paper, we present a novel Deep Triphone Embedding (DTE) representation derived from Deep Neural Network (DNN) to encapsulate the discriminative information present in the adjoining speech frames. DTEs are generated using a four hidden layer DNN with 3000 nodes in each hidden layer at the first-stage. This DNN is trained with the tied-triphone classification accuracy as an optimization criterion. Thereafter, we retain the activation vectors (3000) of the last hidden layer, for each speech MFCC frame, and perform dimension reduction to further obtain a 300 dimensional representation, which we termed as DTE. DTEs along with MFCC features are fed into a second-stage four hidden layer DNN, which is subsequently trained for the task of tied-triphone classification. Both DNNs are trained using tri-phone labels generated from a tied-state triphone HMM-GMM system, by performing a forced-alignment between the transcriptions and MFCC feature frames. We conduct the experiments on publicly available TED-LIUM speech corpus. The results show that the proposed DTE method provides an improvement of absolute 2.11% in phoneme recognition, when compared with a competitive hybrid tied-state triphone HMM-DNN system.
Tasks	Dimensionality Reduction
Published	2017-10-22
URL	http://arxiv.org/abs/1710.07868v2
PDF	http://arxiv.org/pdf/1710.07868v2.pdf
PWC	https://paperswithcode.com/paper/deep-triphone-embedding-improves-phoneme
Repo
Framework

Ideological Sublations: Resolution of Dialectic in Population-based Optimization


Title	Ideological Sublations: Resolution of Dialectic in Population-based Optimization
Authors	S. Hossein Hosseini, Afshin Ebrahimi
Abstract	A population-based optimization algorithm was designed, inspired by two main thinking modes in philosophy, both based on dialectic concept and thesis-antithesis paradigm. They impose two different kinds of dialectics. Idealistic and materialistic antitheses are formulated as optimization models. Based on the models, the population is coordinated for dialectical interactions. At the population-based context, the formulated optimization models are reduced to a simple detection problem for each thinker (particle). According to the assigned thinking mode to each thinker and her/his measurements of corresponding dialectic with other candidate particles, they deterministically decide to interact with a thinker in maximum dialectic with their theses. The position of a thinker at maximum dialectic is known as an available antithesis among the existing solutions. The dialectical interactions at each ideological community are distinguished by meaningful distributions of step-sizes for each thinking mode. In fact, the thinking modes are regarded as exploration and exploitation elements of the proposed algorithm. The result is a delicate balance without any requirement for adjustment of step-size coefficients. Main parameter of the proposed algorithm is the number of particles appointed to each thinking modes, or equivalently for each kind of motions. An additional integer parameter is defined to boost the stability of the final algorithm in some particular problems. The proposed algorithm is evaluated by a testbed of 12 single-objective continuous benchmark functions. Moreover, its performance and speed were highlighted in sparse reconstruction and antenna selection problems, at the context of compressed sensing and massive MIMO, respectively. The results indicate fast and efficient performance in comparison with well-known evolutionary algorithms and dedicated state-of-the-art algorithms.
Tasks
Published	2017-07-21
URL	http://arxiv.org/abs/1707.06992v2
PDF	http://arxiv.org/pdf/1707.06992v2.pdf
PWC	https://paperswithcode.com/paper/ideological-sublations-resolution-of
Repo
Framework

Saliency-based Sequential Image Attention with Multiset Prediction


Title	Saliency-based Sequential Image Attention with Multiset Prediction
Authors	Sean Welleck, Jialin Mao, Kyunghyun Cho, Zheng Zhang
Abstract	Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual architecture that operates on a saliency map and uses a novel attention mechanism to sequentially focus on salient regions and take additional glimpses within those regions. The architecture is motivated by human visual attention, and is used for multi-label image classification on a novel multiset task, demonstrating that it achieves high precision and recall while localizing objects with its attention. Unlike conventional multi-label image classification models, the model supports multiset prediction due to a reinforcement-learning based training process that allows for arbitrary label permutation and multiple instances per label.
Tasks	Image Classification
Published	2017-11-14
URL	http://arxiv.org/abs/1711.05165v1
PDF	http://arxiv.org/pdf/1711.05165v1.pdf
PWC	https://paperswithcode.com/paper/saliency-based-sequential-image-attention
Repo
Framework

Understanding and Detecting Supporting Arguments of Diverse Types


Title	Understanding and Detecting Supporting Arguments of Diverse Types
Authors	Xinyu Hua, Lu Wang
Abstract	We investigate the problem of sentence-level supporting argument detection from relevant documents for user-specified claims. A dataset containing claims and associated citation articles is collected from online debate website idebate.org. We then manually label sentence-level supporting arguments from the documents along with their types as study, factual, opinion, or reasoning. We further characterize arguments of different types, and explore whether leveraging type information can facilitate the supporting arguments detection task. Experimental results show that LambdaMART (Burges, 2010) ranker that uses features informed by argument types yields better performance than the same ranker trained without type information.
Tasks
Published	2017-04-28
URL	http://arxiv.org/abs/1705.00045v2
PDF	http://arxiv.org/pdf/1705.00045v2.pdf
PWC	https://paperswithcode.com/paper/understanding-and-detecting-supporting
Repo
Framework

Is Simple Better? Revisiting Non-linear Matrix Factorization for Learning Incomplete Ratings


Title	Is Simple Better? Revisiting Non-linear Matrix Factorization for Learning Incomplete Ratings
Authors	Vaibhav Krishna, Tian Guo, Nino Antulov-Fantulin
Abstract	Matrix factorization techniques have been widely used as a method for collaborative filtering for recommender systems. In recent times, different variants of deep learning algorithms have been explored in this setting to improve the task of making a personalized recommendation with user-item interaction data. The idea that the mapping between the latent user or item factors and the original features is highly nonlinear suggest that classical matrix factorization techniques are no longer sufficient. In this paper, we propose a multilayer nonlinear semi-nonnegative matrix factorization method, with the motivation that user-item interactions can be modeled more accurately using a linear combination of non-linear item features. Firstly, we learn latent factors for representations of users and items from the designed multilayer nonlinear Semi-NMF approach using explicit ratings. Secondly, the architecture built is compared with deep-learning algorithms like Restricted Boltzmann Machine and state-of-the-art Deep Matrix factorization techniques. By using both supervised rate prediction task and unsupervised clustering in latent item space, we demonstrate that our proposed approach achieves better generalization ability in prediction as well as comparable representation ability as deep matrix factorization in the clustering task.
Tasks	Recommendation Systems
Published	2017-10-16
URL	http://arxiv.org/abs/1710.05613v3
PDF	http://arxiv.org/pdf/1710.05613v3.pdf
PWC	https://paperswithcode.com/paper/is-simple-better-revisiting-non-linear-matrix
Repo
Framework

Hybrid VAE: Improving Deep Generative Models using Partial Observations


Title	Hybrid VAE: Improving Deep Generative Models using Partial Observations
Authors	Sergey Tulyakov, Andrew Fitzgibbon, Sebastian Nowozin
Abstract	Deep neural network models trained on large labeled datasets are the state-of-the-art in a large variety of computer vision tasks. In many applications, however, labeled data is expensive to obtain or requires a time consuming manual annotation process. In contrast, unlabeled data is often abundant and available in large quantities. We present a principled framework to capitalize on unlabeled data by training deep generative models on both labeled and unlabeled data. We show that such a combination is beneficial because the unlabeled data acts as a data-driven form of regularization, allowing generative models trained on few labeled samples to reach the performance of fully-supervised generative models trained on much larger datasets. We call our method Hybrid VAE (H-VAE) as it contains both the generative and the discriminative parts. We validate H-VAE on three large-scale datasets of different modalities: two face datasets: (MultiPIE, CelebA) and a hand pose dataset (NYU Hand Pose). Our qualitative visualizations further support improvements achieved by using partial observations.
Tasks
Published	2017-11-30
URL	http://arxiv.org/abs/1711.11566v1
PDF	http://arxiv.org/pdf/1711.11566v1.pdf
PWC	https://paperswithcode.com/paper/hybrid-vae-improving-deep-generative-models
Repo
Framework

Graph sampling with determinantal processes


Title	Graph sampling with determinantal processes
Authors	Nicolas Tremblay, Pierre-Olivier Amblard, Simon Barthelmé
Abstract	We present a new random sampling strategy for k-bandlimited signals defined on graphs, based on determinantal point processes (DPP). For small graphs, ie, in cases where the spectrum of the graph is accessible, we exhibit a DPP sampling scheme that enables perfect recovery of bandlimited signals. For large graphs, ie, in cases where the graph’s spectrum is not accessible, we investigate, both theoretically and empirically, a sub-optimal but much faster DPP based on loop-erased random walks on the graph. Preliminary experiments show promising results especially in cases where the number of measurements should stay as small as possible and for graphs that have a strong community structure. Our sampling scheme is efficient and can be applied to graphs with up to $10^6$ nodes.
Tasks	Point Processes
Published	2017-03-05
URL	http://arxiv.org/abs/1703.01594v1
PDF	http://arxiv.org/pdf/1703.01594v1.pdf
PWC	https://paperswithcode.com/paper/graph-sampling-with-determinantal-processes
Repo
Framework

Continuous-Time User Modeling in the Presence of Badges: A Probabilistic Approach


Title	Continuous-Time User Modeling in the Presence of Badges: A Probabilistic Approach
Authors	Ali Khodadadi, Seyed Abbas Hosseini, Erfan Tavakoli, Hamid R. Rabiee
Abstract	User modeling plays an important role in delivering customized web services to the users and improving their engagement. However, most user models in the literature do not explicitly consider the temporal behavior of users. More recently, continuous-time user modeling has gained considerable attention and many user behavior models have been proposed based on temporal point processes. However, typical point process based models often considered the impact of peer influence and content on the user participation and neglected other factors. Gamification elements, are among those factors that are neglected, while they have a strong impact on user participation in online services. In this paper, we propose interdependent multi-dimensional temporal point processes that capture the impact of badges on user participation besides the peer influence and content factors. We extend the proposed processes to model user actions over the community based question and answering websites, and propose an inference algorithm based on Variational-EM that can efficiently learn the model parameters. Extensive experiments on both synthetic and real data gathered from Stack Overflow show that our inference algorithm learns the parameters efficiently and the proposed method can better predict the user behavior compared to the alternatives.
Tasks	Point Processes
Published	2017-02-07
URL	http://arxiv.org/abs/1702.01948v1
PDF	http://arxiv.org/pdf/1702.01948v1.pdf
PWC	https://paperswithcode.com/paper/continuous-time-user-modeling-in-the-presence
Repo
Framework

Beyond Uniform Priors in Bayesian Network Structure Learning


Title	Beyond Uniform Priors in Bayesian Network Structure Learning
Authors	Marco Scutari
Abstract	Bayesian network structure learning is often performed in a Bayesian setting, evaluating candidate structures using their posterior probabilities for a given data set. Score-based algorithms then use those posterior probabilities as an objective function and return the maximum a posteriori network as the learned model. For discrete Bayesian networks, the canonical choice for a posterior score is the Bayesian Dirichlet equivalent uniform (BDeu) marginal likelihood with a uniform (U) graph prior, which assumes a uniform prior both on the network structures and on the parameters of the networks. In this paper, we investigate the problems arising from these assumptions, focusing on those caused by small sample sizes and sparse data. We then propose an alternative posterior score: the Bayesian Dirichlet sparse (BDs) marginal likelihood with a marginal uniform (MU) graph prior. Like U+BDeu, MU+BDs does not require any prior information on the probabilistic structure of the data and can be used as a replacement noninformative score. We study its theoretical properties and we evaluate its performance in an extensive simulation study, showing that MU+BDs is both more accurate than U+BDeu in learning the structure of the network and competitive in predicting power, while not being computationally more complex to estimate.
Tasks
Published	2017-04-12
URL	http://arxiv.org/abs/1704.03942v1
PDF	http://arxiv.org/pdf/1704.03942v1.pdf
PWC	https://paperswithcode.com/paper/beyond-uniform-priors-in-bayesian-network
Repo
Framework

Criticality & Deep Learning II: Momentum Renormalisation Group


Title	Criticality & Deep Learning II: Momentum Renormalisation Group
Authors	Dan Oprisa, Peter Toth
Abstract	Guided by critical systems found in nature we develop a novel mechanism consisting of inhomogeneous polynomial regularisation via which we can induce scale invariance in deep learning systems. Technically, we map our deep learning (DL) setup to a genuine field theory, on which we act with the Renormalisation Group (RG) in momentum space and produce the flow equations of the couplings; those are translated to constraints and consequently interpreted as “critical regularisation” conditions in the optimiser; the resulting equations hence prove to be sufficient conditions for - and serve as an elegant and simple mechanism to induce scale invariance in any deep learning setup.
Tasks
Published	2017-05-31
URL	http://arxiv.org/abs/1705.11023v1
PDF	http://arxiv.org/pdf/1705.11023v1.pdf
PWC	https://paperswithcode.com/paper/criticality-deep-learning-ii-momentum
Repo
Framework

A Generalized Recurrent Neural Architecture for Text Classification with Multi-Task Learning


Title	A Generalized Recurrent Neural Architecture for Text Classification with Multi-Task Learning
Authors	Honglun Zhang, Liqiang Xiao, Yongkun Wang, Yaohui Jin
Abstract	Multi-task learning leverages potential correlations among related tasks to extract common features and yield performance gains. However, most previous works only consider simple or weak interactions, thereby failing to model complex correlations among three or more tasks. In this paper, we propose a multi-task learning architecture with four types of recurrent neural layers to fuse information across multiple related tasks. The architecture is structurally flexible and considers various interactions among tasks, which can be regarded as a generalized case of many previous works. Extensive experiments on five benchmark datasets for text classification show that our model can significantly improve performances of related tasks with additional information from others.
Tasks	Multi-Task Learning, Text Classification
Published	2017-07-10
URL	http://arxiv.org/abs/1707.02892v1
PDF	http://arxiv.org/pdf/1707.02892v1.pdf
PWC	https://paperswithcode.com/paper/a-generalized-recurrent-neural-architecture
Repo
Framework

TandemNet: Distilling Knowledge from Medical Images Using Diagnostic Reports as Optional Semantic References


Title	TandemNet: Distilling Knowledge from Medical Images Using Diagnostic Reports as Optional Semantic References
Authors	Zizhao Zhang, Pingjun Chen, Manish Sapkota, Lin Yang
Abstract	In this paper, we introduce the semantic knowledge of medical images from their diagnostic reports to provide an inspirational network training and an interpretable prediction mechanism with our proposed novel multimodal neural network, namely TandemNet. Inside TandemNet, a language model is used to represent report text, which cooperates with the image model in a tandem scheme. We propose a novel dual-attention model that facilitates high-level interactions between visual and semantic information and effectively distills useful features for prediction. In the testing stage, TandemNet can make accurate image prediction with an optional report text input. It also interprets its prediction by producing attention on the image and text informative feature pieces, and further generating diagnostic report paragraphs. Based on a pathological bladder cancer images and their diagnostic reports (BCIDR) dataset, sufficient experiments demonstrate that our method effectively learns and integrates knowledge from multimodalities and obtains significantly improved performance than comparing baselines.
Tasks	Language Modelling
Published	2017-08-10
URL	http://arxiv.org/abs/1708.03070v1
PDF	http://arxiv.org/pdf/1708.03070v1.pdf
PWC	https://paperswithcode.com/paper/tandemnet-distilling-knowledge-from-medical
Repo
Framework