May 6, 2019

3282 words 16 mins read

Paper Group ANR 289

Completely random measures for modeling power laws in sparse graphs. Resolution- and throughput-enhanced spectroscopy using high-throughput computational slit. Question Answering via Integer Programming over Semi-Structured Knowledge. Robust end-to-end deep audiovisual speech recognition. Deep Transfer Learning with Joint Adaptation Networks. Fast …

Completely random measures for modeling power laws in sparse graphs


Title	Completely random measures for modeling power laws in sparse graphs
Authors	Diana Cai, Tamara Broderick
Abstract	Network data appear in a number of applications, such as online social networks and biological networks, and there is growing interest in both developing models for networks as well as studying the properties of such data. Since individual network datasets continue to grow in size, it is necessary to develop models that accurately represent the real-life scaling properties of networks. One behavior of interest is having a power law in the degree distribution. However, other types of power laws that have been observed empirically and considered for applications such as clustering and feature allocation models have not been studied as frequently in models for graph data. In this paper, we enumerate desirable asymptotic behavior that may be of interest for modeling graph data, including sparsity and several types of power laws. We outline a general framework for graph generative models using completely random measures; by contrast to the pioneering work of Caron and Fox (2015), we consider instantiating more of the existing atoms of the random measure as the dataset size increases rather than adding new atoms to the measure. We see that these two models can be complementary; they respectively yield interpretations as (1) time passing among existing members of a network and (2) new individuals joining a network. We detail a particular instance of this framework and show simulated results that suggest this model exhibits some desirable asymptotic power-law behavior.
Tasks
Published	2016-03-22
URL	http://arxiv.org/abs/1603.06915v1
PDF	http://arxiv.org/pdf/1603.06915v1.pdf
PWC	https://paperswithcode.com/paper/completely-random-measures-for-modeling-power
Repo
Framework

Resolution- and throughput-enhanced spectroscopy using high-throughput computational slit


Title	Resolution- and throughput-enhanced spectroscopy using high-throughput computational slit
Authors	Farnoud Kazemzadeh, Alexander Wong
Abstract	There exists a fundamental tradeoff between spectral resolution and the efficiency or throughput for all optical spectrometers. The primary factors affecting the spectral resolution and throughput of an optical spectrometer are the size of the entrance aperture and the optical power of the focusing element. Thus far collective optimization of the above mentioned has proven difficult. Here, we introduce the concept of high-throughput computational slits (HTCS), a numerical technique for improving both the effective spectral resolution and efficiency of a spectrometer. The proposed HTCS approach was experimentally validated using an optical spectrometer configured with a 200 um entrance aperture, test, and a 50 um entrance aperture, control, demonstrating improvements in spectral resolution of the spectrum by ~ 50% over the control spectral resolution and improvements in efficiency of > 2 times over the efficiency of the largest entrance aperture used in the study while producing highly accurate spectra.
Tasks
Published	2016-06-29
URL	http://arxiv.org/abs/1606.09072v2
PDF	http://arxiv.org/pdf/1606.09072v2.pdf
PWC	https://paperswithcode.com/paper/resolution-and-throughput-enhanced
Repo
Framework

Question Answering via Integer Programming over Semi-Structured Knowledge


Title	Question Answering via Integer Programming over Semi-Structured Knowledge
Authors	Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Peter Clark, Oren Etzioni, Dan Roth
Abstract	Answering science questions posed in natural language is an important AI challenge. Answering such questions often requires non-trivial inference and knowledge that goes beyond factoid retrieval. Yet, most systems for this task are based on relatively shallow Information Retrieval (IR) and statistical correlation techniques operating on large unstructured corpora. We propose a structured inference system for this task, formulated as an Integer Linear Program (ILP), that answers natural language questions using a semi-structured knowledge base derived from text, including questions requiring multi-step inference and a combination of multiple facts. On a dataset of real, unseen science questions, our system significantly outperforms (+14%) the best previous attempt at structured reasoning for this task, which used Markov Logic Networks (MLNs). It also improves upon a previous ILP formulation by 17.7%. When combined with unstructured inference methods, the ILP system significantly boosts overall performance (+10%). Finally, we show our approach is substantially more robust to a simple answer perturbation compared to statistical correlation methods.
Tasks	Information Retrieval, Question Answering
Published	2016-04-20
URL	http://arxiv.org/abs/1604.06076v1
PDF	http://arxiv.org/pdf/1604.06076v1.pdf
PWC	https://paperswithcode.com/paper/question-answering-via-integer-programming
Repo
Framework

Robust end-to-end deep audiovisual speech recognition


Title	Robust end-to-end deep audiovisual speech recognition
Authors	Ramon Sanabria, Florian Metze, Fernando De La Torre
Abstract	Speech is one of the most effective ways of communication among humans. Even though audio is the most common way of transmitting speech, very important information can be found in other modalities, such as vision. Vision is particularly useful when the acoustic signal is corrupted. Multi-modal speech recognition however has not yet found wide-spread use, mostly because the temporal alignment and fusion of the different information sources is challenging. This paper presents an end-to-end audiovisual speech recognizer (AVSR), based on recurrent neural networks (RNN) with a connectionist temporal classification (CTC) loss function. CTC creates sparse “peaky” output activations, and we analyze the differences in the alignments of output targets (phonemes or visemes) between audio-only, video-only, and audio-visual feature representations. We present the first such experiments on the large vocabulary IBM ViaVoice database, which outperform previously published approaches on phone accuracy in clean and noisy conditions.
Tasks	Speech Recognition
Published	2016-11-21
URL	http://arxiv.org/abs/1611.06986v1
PDF	http://arxiv.org/pdf/1611.06986v1.pdf
PWC	https://paperswithcode.com/paper/robust-end-to-end-deep-audiovisual-speech
Repo
Framework

Deep Transfer Learning with Joint Adaptation Networks


Title	Deep Transfer Learning with Joint Adaptation Networks
Authors	Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
Abstract	Deep networks have been successfully applied to learn transferable features for adapting models from a source domain to a different target domain. In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domains based on a joint maximum mean discrepancy (JMMD) criterion. Adversarial training strategy is adopted to maximize JMMD such that the distributions of the source and target domains are made more distinguishable. Learning can be performed by stochastic gradient descent with the gradients computed by back-propagation in linear-time. Experiments testify that our model yields state of the art results on standard datasets.
Tasks	Transfer Learning
Published	2016-05-21
URL	http://arxiv.org/abs/1605.06636v2
PDF	http://arxiv.org/pdf/1605.06636v2.pdf
PWC	https://paperswithcode.com/paper/deep-transfer-learning-with-joint-adaptation
Repo
Framework

Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets


Title	Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets
Authors	Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, Frank Hutter
Abstract	Bayesian optimization has become a successful tool for hyperparameter optimization of machine learning algorithms, such as support vector machines or deep neural networks. Despite its success, for large datasets, training and validating a single configuration often takes hours, days, or even weeks, which limits the achievable performance. To accelerate hyperparameter optimization, we propose a generative model for the validation error as a function of training set size, which is learned during the optimization process and allows exploration of preliminary configurations on small subsets, by extrapolating to the full dataset. We construct a Bayesian optimization procedure, dubbed Fabolas, which models loss and training time as a function of dataset size and automatically trades off high information gain about the global optimum against computational cost. Experiments optimizing support vector machines and deep neural networks show that Fabolas often finds high-quality solutions 10 to 100 times faster than other state-of-the-art Bayesian optimization methods or the recently proposed bandit strategy Hyperband.
Tasks	Hyperparameter Optimization
Published	2016-05-23
URL	http://arxiv.org/abs/1605.07079v2
PDF	http://arxiv.org/pdf/1605.07079v2.pdf
PWC	https://paperswithcode.com/paper/fast-bayesian-optimization-of-machine
Repo
Framework

Ensemble-Compression: A New Method for Parallel Training of Deep Neural Networks


Title	Ensemble-Compression: A New Method for Parallel Training of Deep Neural Networks
Authors	Shizhao Sun, Wei Chen, Jiang Bian, Xiaoguang Liu, Tie-Yan Liu
Abstract	Parallelization framework has become a necessity to speed up the training of deep neural networks (DNN) recently. Such framework typically employs the Model Average approach, denoted as MA-DNN, in which parallel workers conduct respective training based on their own local data while the parameters of local models are periodically communicated and averaged to obtain a global model which serves as the new start of local models. However, since DNN is a highly non-convex model, averaging parameters cannot ensure that such global model can perform better than those local models. To tackle this problem, we introduce a new parallel training framework called Ensemble-Compression, denoted as EC-DNN. In this framework, we propose to aggregate the local models by ensemble, i.e., averaging the outputs of local models instead of the parameters. As most of prevalent loss functions are convex to the output of DNN, the performance of ensemble-based global model is guaranteed to be at least as good as the average performance of local models. However, a big challenge lies in the explosion of model size since each round of ensemble can give rise to multiple times size increment. Thus, we carry out model compression after each ensemble, specialized by a distillation based method in this paper, to reduce the size of the global model to be the same as the local ones. Our experimental results demonstrate the prominent advantage of EC-DNN over MA-DNN in terms of both accuracy and speedup.
Tasks	Model Compression
Published	2016-06-02
URL	http://arxiv.org/abs/1606.00575v2
PDF	http://arxiv.org/pdf/1606.00575v2.pdf
PWC	https://paperswithcode.com/paper/ensemble-compression-a-new-method-for
Repo
Framework

Humans and deep networks largely agree on which kinds of variation make object recognition harder


Title	Humans and deep networks largely agree on which kinds of variation make object recognition harder
Authors	Saeed Reza Kheradpisheh, Masoud Ghodrati, Mohammad Ganjtabesh, Timothée Masquelier
Abstract	View-invariant object recognition is a challenging problem, which has attracted much attention among the psychology, neuroscience, and computer vision communities. Humans are notoriously good at it, even if some variations are presumably more difficult to handle than others (e.g. 3D rotations). Humans are thought to solve the problem through hierarchical processing along the ventral stream, which progressively extracts more and more invariant visual features. This feed-forward architecture has inspired a new generation of bio-inspired computer vision systems called deep convolutional neural networks (DCNN), which are currently the best algorithms for object recognition in natural images. Here, for the first time, we systematically compared human feed-forward vision and DCNNs at view-invariant object recognition using the same images and controlling for both the kinds of transformation as well as their magnitude. We used four object categories and images were rendered from 3D computer models. In total, 89 human subjects participated in 10 experiments in which they had to discriminate between two or four categories after rapid presentation with backward masking. We also tested two recent DCNNs on the same tasks. We found that humans and DCNNs largely agreed on the relative difficulties of each kind of variation: rotation in depth is by far the hardest transformation to handle, followed by scale, then rotation in plane, and finally position. This suggests that humans recognize objects mainly through 2D template matching, rather than by constructing 3D object models, and that DCNNs are not too unreasonable models of human feed-forward vision. Also, our results show that the variation levels in rotation in depth and scale strongly modulate both humans’ and DCNNs’ recognition performances. We thus argue that these variations should be controlled in the image datasets used in vision research.
Tasks	Object Recognition
Published	2016-04-21
URL	http://arxiv.org/abs/1604.06486v1
PDF	http://arxiv.org/pdf/1604.06486v1.pdf
PWC	https://paperswithcode.com/paper/humans-and-deep-networks-largely-agree-on
Repo
Framework

Positive blood culture detection in time series data using a BiLSTM network


Title	Positive blood culture detection in time series data using a BiLSTM network
Authors	Leen De Baets, Joeri Ruyssinck, Thomas Peiffer, Johan Decruyenaere, Filip De Turck, Femke Ongenae, Tom Dhaene
Abstract	The presence of bacteria or fungi in the bloodstream of patients is abnormal and can lead to life-threatening conditions. A computational model based on a bidirectional long short-term memory artificial neural network, is explored to assist doctors in the intensive care unit to predict whether examination of blood cultures of patients will return positive. As input it uses nine monitored clinical parameters, presented as time series data, collected from 2177 ICU admissions at the Ghent University Hospital. Our main goal is to determine if general machine learning methods and more specific, temporal models, can be used to create an early detection system. This preliminary research obtains an area of 71.95% under the precision recall curve, proving the potential of temporal neural networks in this context.
Tasks	Time Series
Published	2016-12-03
URL	http://arxiv.org/abs/1612.00962v1
PDF	http://arxiv.org/pdf/1612.00962v1.pdf
PWC	https://paperswithcode.com/paper/positive-blood-culture-detection-in-time
Repo
Framework

Robust Method of Vote Aggregation and Proposition Verification for Invariant Local Features


Title	Robust Method of Vote Aggregation and Proposition Verification for Invariant Local Features
Authors	Grzegorz Kurzejamski, Jacek Zawistowski, Grzegorz Sarwas
Abstract	This paper presents a method for analysis of the vote space created from the local features extraction process in a multi-detection system. The method is opposed to the classic clustering approach and gives a high level of control over the clusters composition for further verification steps. Proposed method comprises of the graphical vote space presentation, the proposition generation, the two-pass iterative vote aggregation and the cascade filters for verification of the propositions. Cascade filters contain all of the minor algorithms needed for effective object detection verification. The new approach does not have the drawbacks of the classic clustering approaches and gives a substantial control over process of detection. Method exhibits an exceptionally high detection rate in conjunction with a low false detection chance in comparison to alternative methods.
Tasks	Object Detection
Published	2016-01-05
URL	http://arxiv.org/abs/1601.00781v1
PDF	http://arxiv.org/pdf/1601.00781v1.pdf
PWC	https://paperswithcode.com/paper/robust-method-of-vote-aggregation-and
Repo
Framework

A simple method for estimating the fractal dimension from digital images: The compression dimension


Title	A simple method for estimating the fractal dimension from digital images: The compression dimension
Authors	P. Chamorro-Posada
Abstract	The fractal structure of real world objects is often analyzed using digital images. In this context, the compression fractal dimension is put forward. It provides a simple method for the direct estimation of the dimension of fractals stored as digital image files. The computational scheme can be implemented using readily available free software. Its simplicity also makes it very interesting for introductory elaborations of basic concepts of fractal geometry, complexity, and information theory. A test of the computational scheme using limited-quality images of well-defined fractal sets obtained from the Internet and free software has been performed. Also, a systematic evaluation of the proposed method using computer generated images of the Weierstrass cosine function shows an accuracy comparable to those of the methods most commonly used to estimate the dimension of fractal data sequences applied to the same test problem.
Tasks
Published	2016-02-03
URL	http://arxiv.org/abs/1602.02139v2
PDF	http://arxiv.org/pdf/1602.02139v2.pdf
PWC	https://paperswithcode.com/paper/a-simple-method-for-estimating-the-fractal
Repo
Framework

Deep Learning of Appearance Models for Online Object Tracking


Title	Deep Learning of Appearance Models for Online Object Tracking
Authors	Mengyao Zhai, Mehrsan Javan Roshtkhari, Greg Mori
Abstract	This paper introduces a novel deep learning based approach for vision based single target tracking. We address this problem by proposing a network architecture which takes the input video frames and directly computes the tracking score for any candidate target location by estimating the probability distributions of the positive and negative examples. This is achieved by combining a deep convolutional neural network with a Bayesian loss layer in a unified framework. In order to deal with the limited number of positive training examples, the network is pre-trained offline for a generic image feature representation and then is fine-tuned in multiple steps. An online fine-tuning step is carried out at every frame to learn the appearance of the target. We adopt a two-stage iterative algorithm to adaptively update the network parameters and maintain a probability density for target/non-target regions. The tracker has been tested on the standard tracking benchmark and the results indicate that the proposed solution achieves state-of-the-art tracking results.
Tasks	Object Tracking
Published	2016-07-09
URL	http://arxiv.org/abs/1607.02568v1
PDF	http://arxiv.org/pdf/1607.02568v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-of-appearance-models-for-online
Repo
Framework

Spectral Learning for Supervised Topic Models


Title	Spectral Learning for Supervised Topic Models
Authors	Yong Ren, Yining Wang, Jun Zhu
Abstract	Supervised topic models simultaneously model the latent topic structure of large collections of documents and a response variable associated with each document. Existing inference methods are based on variational approximation or Monte Carlo sampling, which often suffers from the local minimum defect. Spectral methods have been applied to learn unsupervised topic models, such as latent Dirichlet allocation (LDA), with provable guarantees. This paper investigates the possibility of applying spectral methods to recover the parameters of supervised LDA (sLDA). We first present a two-stage spectral method, which recovers the parameters of LDA followed by a power update method to recover the regression model parameters. Then, we further present a single-phase spectral algorithm to jointly recover the topic distribution matrix as well as the regression weights. Our spectral algorithms are provably correct and computationally efficient. We prove a sample complexity bound for each algorithm and subsequently derive a sufficient condition for the identifiability of sLDA. Thorough experiments on synthetic and real-world datasets verify the theory and demonstrate the practical effectiveness of the spectral algorithms. In fact, our results on a large-scale review rating dataset demonstrate that our single-phase spectral algorithm alone gets comparable or even better performance than state-of-the-art methods, while previous work on spectral methods has rarely reported such promising performance.
Tasks	Topic Models
Published	2016-02-19
URL	http://arxiv.org/abs/1602.06025v1
PDF	http://arxiv.org/pdf/1602.06025v1.pdf
PWC	https://paperswithcode.com/paper/spectral-learning-for-supervised-topic-models
Repo
Framework

Towards a continuous modeling of natural language domains


Title	Towards a continuous modeling of natural language domains
Authors	Sebastian Ruder, Parsa Ghaffari, John G. Breslin
Abstract	Humans continuously adapt their style and language to a variety of domains. However, a reliable definition of `domain’ has eluded researchers thus far. Additionally, the notion of discrete domains stands in contrast to the multiplicity of heterogeneous domains that humans navigate, many of which overlap. In order to better understand the change and variation of human language, we draw on research in domain adaptation and extend the notion of discrete domains to the continuous spectrum. We propose representation learning-based models that can adapt to continuous domains and detail how these can be used to investigate variation in language. To this end, we propose to use dialogue modeling as a test bed due to its proximity to language modeling and its social component. \|
Tasks	Domain Adaptation, Language Modelling, Representation Learning
Published	2016-10-28
URL	http://arxiv.org/abs/1610.09158v1
PDF	http://arxiv.org/pdf/1610.09158v1.pdf
PWC	https://paperswithcode.com/paper/towards-a-continuous-modeling-of-natural
Repo
Framework

Proceedings of the LexSem+Logics Workshop 2016


Title	Proceedings of the LexSem+Logics Workshop 2016
Authors	Steven Neale, Valeria de Paiva, Arantxa Otegi, Alexandre Rademaker
Abstract	Lexical semantics continues to play an important role in driving research directions in NLP, with the recognition and understanding of context becoming increasingly important in delivering successful outcomes in NLP tasks. Besides traditional processing areas such as word sense and named entity disambiguation, the creation and maintenance of dictionaries, annotated corpora and resources have become cornerstones of lexical semantics research and produced a wealth of contextual information that NLP processes can exploit. New efforts both to link and construct from scratch such information - as Linked Open Data or by way of formal tools coming from logic, ontologies and automated reasoning - have increased the interoperability and accessibility of resources for lexical and computational semantics, even in those languages for which they have previously been limited. LexSem+Logics 2016 combines the 1st Workshop on Lexical Semantics for Lesser-Resources Languages and the 3rd Workshop on Logics and Ontologies. The accepted papers in our program covered topics across these two areas, including: the encoding of plurals in Wordnets, the creation of a thesaurus from multiple sources based on semantic similarity metrics, and the use of cross-lingual treebanks and annotations for universal part-of-speech tagging. We also welcomed talks from two distinguished speakers: on Portuguese lexical knowledge bases (different approaches, results and their application in NLP tasks) and on new strategies for open information extraction (the capture of verb-based propositions from massive text corpora).
Tasks	Entity Disambiguation, Open Information Extraction, Part-Of-Speech Tagging, Semantic Similarity, Semantic Textual Similarity
Published	2016-08-14
URL	http://arxiv.org/abs/1608.04767v1
PDF	http://arxiv.org/pdf/1608.04767v1.pdf
PWC	https://paperswithcode.com/paper/proceedings-of-the-lexsemlogics-workshop-2016
Repo
Framework