May 7, 2019

3062 words 15 mins read

Paper Group AWR 49

Discriminating sample groups with multi-way data. A Generic Inverted Index Framework for Similarity Search on the GPU - Technical Report. Fast Supervised Discrete Hashing and its Analysis. NFL Play Prediction. Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests. Wasserstein Discriminant Analysis. A Study of MatchPyramid Models on Ad …

Discriminating sample groups with multi-way data


Title	Discriminating sample groups with multi-way data
Authors	Tianmeng Lyu, Eric F. Lock, Lynn E. Eberly
Abstract	High-dimensional linear classifiers, such as the support vector machine (SVM) and distance weighted discrimination (DWD), are commonly used in biomedical research to distinguish groups of subjects based on a large number of features. However, their use is limited to applications where a single vector of features is measured for each subject. In practice data are often multi-way, or measured over multiple dimensions. For example, metabolite abundance may be measured over multiple regions or tissues, or gene expression may be measured over multiple time points, for the same subjects. We propose a framework for linear classification of high-dimensional multi-way data, in which coefficients can be factorized into weights that are specific to each dimension. More generally, the coefficients for each measurement in a multi-way dataset are assumed to have low-rank structure. This framework extends existing classification techniques, and we have implemented multi-way versions of SVM and DWD. We describe informative simulation results, and apply multi-way DWD to data for two very different clinical research studies. The first study uses metabolite magnetic resonance spectroscopy data over multiple brain regions to compare patients with and without spinocerebellar ataxia, the second uses publicly available gene expression time-course data to compare treatment responses for patients with multiple sclerosis. Our method improves performance and simplifies interpretation over naive applications of full rank linear classification to multi-way data. An R package is available at https://github.com/lockEF/MultiwayClassification .
Tasks
Published	2016-06-26
URL	http://arxiv.org/abs/1606.08046v1
PDF	http://arxiv.org/pdf/1606.08046v1.pdf
PWC	https://paperswithcode.com/paper/discriminating-sample-groups-with-multi-way
Repo	https://github.com/lockEF/MultiwayClassification
Framework	none

A Generic Inverted Index Framework for Similarity Search on the GPU - Technical Report


Title	A Generic Inverted Index Framework for Similarity Search on the GPU - Technical Report
Authors	Jingbo Zhou, Qi Guo, H. V. Jagadish, Luboš Krčál, Siyuan Liu, Wenhao Luan, Anthony K. H. Tung, Yueji Yang, Yuxin Zheng
Abstract	We propose a novel generic inverted index framework on the GPU (called GENIE), aiming to reduce the programming complexity of the GPU for parallel similarity search of different data types. Not every data type and similarity measure are supported by GENIE, but many popular ones are. We present the system design of GENIE, and demonstrate similarity search with GENIE on several data types along with a theoretical analysis of search results. A new concept of locality sensitive hashing (LSH) named $\tau$-ANN search, and a novel data structure c-PQ on the GPU are also proposed for achieving this purpose. Extensive experiments on different real-life datasets demonstrate the efficiency and effectiveness of our framework. The implemented system has been released as open source.
Tasks
Published	2016-03-28
URL	http://arxiv.org/abs/1603.08390v3
PDF	http://arxiv.org/pdf/1603.08390v3.pdf
PWC	https://paperswithcode.com/paper/a-generic-inverted-index-framework-for
Repo	https://github.com/SeSaMe-NUS/genie
Framework	none

Fast Supervised Discrete Hashing and its Analysis


Title	Fast Supervised Discrete Hashing and its Analysis
Authors	Gou Koutaki, Keiichiro Shirai, Mitsuru Ambai
Abstract	In this paper, we propose a learning-based supervised discrete hashing method. Binary hashing is widely used for large-scale image retrieval as well as video and document searches because the compact representation of binary code is essential for data storage and reasonable for query searches using bit-operations. The recently proposed Supervised Discrete Hashing (SDH) efficiently solves mixed-integer programming problems by alternating optimization and the Discrete Cyclic Coordinate descent (DCC) method. We show that the SDH model can be simplified without performance degradation based on some preliminary experiments; we call the approximate model for this the “Fast SDH” (FSDH) model. We analyze the FSDH model and provide a mathematically exact solution for it. In contrast to SDH, our model does not require an alternating optimization algorithm and does not depend on initial values. FSDH is also easier to implement than Iterative Quantization (ITQ). Experimental results involving a large-scale database showed that FSDH outperforms conventional SDH in terms of precision, recall, and computation time.
Tasks	Image Retrieval, Quantization
Published	2016-11-30
URL	http://arxiv.org/abs/1611.10017v1
PDF	http://arxiv.org/pdf/1611.10017v1.pdf
PWC	https://paperswithcode.com/paper/fast-supervised-discrete-hashing-and-its
Repo	https://github.com/goukoutaki/FSDH
Framework	none

NFL Play Prediction


Title	NFL Play Prediction
Authors	Brendan Teich, Roman Lutz, Valentin Kassarnig
Abstract	Based on NFL game data we try to predict the outcome of a play in multiple different ways. An application of this is the following: by plugging in various play options one could determine the best play for a given situation in real time. While the outcome of a play can be described in many ways we had the most promising results with a newly defined measure that we call “progress”. We see this work as a first step to include predictive analysis into NFL playcalling.
Tasks
Published	2016-01-04
URL	http://arxiv.org/abs/1601.00574v1
PDF	http://arxiv.org/pdf/1601.00574v1.pdf
PWC	https://paperswithcode.com/paper/nfl-play-prediction
Repo	https://github.com/romanlutz/NFLPlayPrediction
Framework	none

Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests


Title	Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests
Authors	Narges Razavian, Jake Marcus, David Sontag
Abstract	Disparate areas of machine learning have benefited from models that can take raw data with little preprocessing as input and learn rich representations of that raw data in order to perform well on a given prediction task. We evaluate this approach in healthcare by using longitudinal measurements of lab tests, one of the more raw signals of a patient’s health state widely available in clinical data, to predict disease onsets. In particular, we train a Long Short-Term Memory (LSTM) recurrent neural network and two novel convolutional neural networks for multi-task prediction of disease onset for 133 conditions based on 18 common lab tests measured over time in a cohort of 298K patients derived from 8 years of administrative claims data. We compare the neural networks to a logistic regression with several hand-engineered, clinically relevant features. We find that the representation-based learning approaches significantly outperform this baseline. We believe that our work suggests a new avenue for patient risk stratification based solely on lab results.
Tasks
Published	2016-08-02
URL	http://arxiv.org/abs/1608.00647v3
PDF	http://arxiv.org/pdf/1608.00647v3.pdf
PWC	https://paperswithcode.com/paper/multi-task-prediction-of-disease-onsets-from
Repo	https://github.com/clinicalml/deepDiagnosis
Framework	torch

Wasserstein Discriminant Analysis


Title	Wasserstein Discriminant Analysis
Authors	Rémi Flamary, Marco Cuturi, Nicolas Courty, Alain Rakotomamonjy
Abstract	Wasserstein Discriminant Analysis (WDA) is a new supervised method that can improve classification of high-dimensional data by computing a suitable linear map onto a lower dimensional subspace. Following the blueprint of classical Linear Discriminant Analysis (LDA), WDA selects the projection matrix that maximizes the ratio of two quantities: the dispersion of projected points coming from different classes, divided by the dispersion of projected points coming from the same class. To quantify dispersion, WDA uses regularized Wasserstein distances, rather than cross-variance measures which have been usually considered, notably in LDA. Thanks to the the underlying principles of optimal transport, WDA is able to capture both global (at distribution scale) and local (at samples scale) interactions between classes. Regularized Wasserstein distances can be computed using the Sinkhorn matrix scaling algorithm; We show that the optimization of WDA can be tackled using automatic differentiation of Sinkhorn iterations. Numerical experiments show promising results both in terms of prediction and visualization on toy examples and real life datasets such as MNIST and on deep features obtained from a subset of the Caltech dataset.
Tasks
Published	2016-08-29
URL	http://arxiv.org/abs/1608.08063v2
PDF	http://arxiv.org/pdf/1608.08063v2.pdf
PWC	https://paperswithcode.com/paper/wasserstein-discriminant-analysis
Repo	https://github.com/rflamary/POT
Framework	none

A Study of MatchPyramid Models on Ad-hoc Retrieval


Title	A Study of MatchPyramid Models on Ad-hoc Retrieval
Authors	Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Xueqi Cheng
Abstract	Deep neural networks have been successfully applied to many text matching tasks, such as paraphrase identification, question answering, and machine translation. Although ad-hoc retrieval can also be formalized as a text matching task, few deep models have been tested on it. In this paper, we study a state-of-the-art deep matching model, namely MatchPyramid, on the ad-hoc retrieval task. The MatchPyramid model employs a convolutional neural network over the interactions between query and document to produce the matching score. We conducted extensive experiments to study the impact of different pooling sizes, interaction functions and kernel sizes on the retrieval performance. Finally, we show that the MatchPyramid models can significantly outperform several recently introduced deep matching models on the retrieval task, but still cannot compete with the traditional retrieval models, such as BM25 and language models.
Tasks	Machine Translation, Paraphrase Identification, Question Answering, Text Matching
Published	2016-06-15
URL	https://arxiv.org/abs/1606.04648v1
PDF	https://arxiv.org/pdf/1606.04648v1.pdf
PWC	https://paperswithcode.com/paper/a-study-of-matchpyramid-models-on-ad-hoc
Repo	https://github.com/albpurpura/PE4IR
Framework	pytorch

Automating biomedical data science through tree-based pipeline optimization


Title	Automating biomedical data science through tree-based pipeline optimization
Authors	Randal S. Olson, Ryan J. Urbanowicz, Peter C. Andrews, Nicole A. Lavender, La Creis Kidd, Jason H. Moore
Abstract	Over the past decade, data science and machine learning has grown from a mysterious art form to a staple tool across a variety of fields in academia, business, and government. In this paper, we introduce the concept of tree-based pipeline optimization for automating one of the most tedious parts of machine learning—pipeline design. We implement a Tree-based Pipeline Optimization Tool (TPOT) and demonstrate its effectiveness on a series of simulated and real-world genetic data sets. In particular, we show that TPOT can build machine learning pipelines that achieve competitive classification accuracy and discover novel pipeline operators—such as synthetic feature constructors—that significantly improve classification accuracy on these data sets. We also highlight the current challenges to pipeline optimization, such as the tendency to produce pipelines that overfit the data, and suggest future research paths to overcome these challenges. As such, this work represents an early step toward fully automating machine learning pipeline design.
Tasks	Hyperparameter Optimization
Published	2016-01-28
URL	http://arxiv.org/abs/1601.07925v1
PDF	http://arxiv.org/pdf/1601.07925v1.pdf
PWC	https://paperswithcode.com/paper/automating-biomedical-data-science-through
Repo	https://github.com/rhiever/tpot
Framework	none

shapeDTW: shape Dynamic Time Warping


Title	shapeDTW: shape Dynamic Time Warping
Authors	Jiaping Zhao, Laurent Itti
Abstract	Dynamic Time Warping (DTW) is an algorithm to align temporal sequences with possible local non-linear distortions, and has been widely applied to audio, video and graphics data alignments. DTW is essentially a point-to-point matching method under some boundary and temporal consistency constraints. Although DTW obtains a global optimal solution, it does not necessarily achieve locally sensible matchings. Concretely, two temporal points with entirely dissimilar local structures may be matched by DTW. To address this problem, we propose an improved alignment algorithm, named shape Dynamic Time Warping (shapeDTW), which enhances DTW by taking point-wise local structural information into consideration. shapeDTW is inherently a DTW algorithm, but additionally attempts to pair locally similar structures and to avoid matching points with distinct neighborhood structures. We apply shapeDTW to align audio signal pairs having ground-truth alignments, as well as artificially simulated pairs of aligned sequences, and obtain quantitatively much lower alignment errors than DTW and its two variants. When shapeDTW is used as a distance measure in a nearest neighbor classifier (NN-shapeDTW) to classify time series, it beats DTW on 64 out of 84 UCR time series datasets, with significantly improved classification accuracies. By using a properly designed local structure descriptor, shapeDTW improves accuracies by more than 10% on 18 datasets. To the best of our knowledge, shapeDTW is the first distance measure under the nearest neighbor classifier scheme to significantly outperform DTW, which had been widely recognized as the best distance measure to date. Our code is publicly accessible at: https://github.com/jiapingz/shapeDTW.
Tasks	Time Series
Published	2016-06-06
URL	http://arxiv.org/abs/1606.01601v1
PDF	http://arxiv.org/pdf/1606.01601v1.pdf
PWC	https://paperswithcode.com/paper/shapedtw-shape-dynamic-time-warping
Repo	https://github.com/9552nZ/SmartSheetMusic
Framework	none

Context Encoders: Feature Learning by Inpainting


Title	Context Encoders: Feature Learning by Inpainting
Authors	Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, Alexei A. Efros
Abstract	We present an unsupervised visual feature learning algorithm driven by context-based pixel prediction. By analogy with auto-encoders, we propose Context Encoders – a convolutional neural network trained to generate the contents of an arbitrary image region conditioned on its surroundings. In order to succeed at this task, context encoders need to both understand the content of the entire image, as well as produce a plausible hypothesis for the missing part(s). When training context encoders, we have experimented with both a standard pixel-wise reconstruction loss, as well as a reconstruction plus an adversarial loss. The latter produces much sharper results because it can better handle multiple modes in the output. We found that a context encoder learns a representation that captures not just appearance but also the semantics of visual structures. We quantitatively demonstrate the effectiveness of our learned features for CNN pre-training on classification, detection, and segmentation tasks. Furthermore, context encoders can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods.
Tasks
Published	2016-04-25
URL	http://arxiv.org/abs/1604.07379v2
PDF	http://arxiv.org/pdf/1604.07379v2.pdf
PWC	https://paperswithcode.com/paper/context-encoders-feature-learning-by
Repo	https://github.com/pdway53/GAN_Food_image_impair
Framework	pytorch

Neural Architectures for Fine-grained Entity Type Classification


Title	Neural Architectures for Fine-grained Entity Type Classification
Authors	Sonse Shimaoka, Pontus Stenetorp, Kentaro Inui, Sebastian Riedel
Abstract	In this work, we investigate several neural network architectures for fine-grained entity type classification. Particularly, we consider extensions to a recently proposed attentive neural architecture and make three key contributions. Previous work on attentive neural architectures do not consider hand-crafted features, we combine learnt and hand-crafted features and observe that they complement each other. Additionally, through quantitative analysis we establish that the attention mechanism is capable of learning to attend over syntactic heads and the phrase containing the mention, where both are known strong hand-crafted features for our task. We enable parameter sharing through a hierarchical label encoding method, that in low-dimensional projections show clear clusters for each type hierarchy. Lastly, despite using the same evaluation dataset, the literature frequently compare models trained using different data. We establish that the choice of training data has a drastic impact on performance, with decreases by as much as 9.85% loose micro F1 score for a previously proposed method. Despite this, our best model achieves state-of-the-art results with 75.36% loose micro F1 score on the well- established FIGER (GOLD) dataset.
Tasks
Published	2016-06-04
URL	http://arxiv.org/abs/1606.01341v2
PDF	http://arxiv.org/pdf/1606.01341v2.pdf
PWC	https://paperswithcode.com/paper/neural-architectures-for-fine-grained-entity
Repo	https://github.com/shimaokasonse/NFGEC
Framework	tf

Ladder Variational Autoencoders


Title	Ladder Variational Autoencoders
Authors	Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther
Abstract	Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that recursively corrects the generative distribution by a data dependent approximate likelihood in a process resembling the recently proposed Ladder Network. We show that this model provides state of the art predictive log-likelihood and tighter log-likelihood lower bound compared to the purely bottom-up inference in layered Variational Autoencoders and other generative models. We provide a detailed analysis of the learned hierarchical latent representation and show that our new inference model is qualitatively different and utilizes a deeper more distributed hierarchy of latent variables. Finally, we observe that batch normalization and deterministic warm-up (gradually turning on the KL-term) are crucial for training variational models with many stochastic layers.
Tasks
Published	2016-02-06
URL	http://arxiv.org/abs/1602.02282v3
PDF	http://arxiv.org/pdf/1602.02282v3.pdf
PWC	https://paperswithcode.com/paper/ladder-variational-autoencoders
Repo	https://github.com/unsuthee/VariationalDeepSemanticHashing
Framework	tf

Keyframe-based monocular SLAM: design, survey, and future directions


Title	Keyframe-based monocular SLAM: design, survey, and future directions
Authors	Georges Younes, Daniel Asmar, Elie Shammas, John Zelek
Abstract	Extensive research in the field of monocular SLAM for the past fifteen years has yielded workable systems that found their way into various applications in robotics and augmented reality. Although filter-based monocular SLAM systems were common at some time, the more efficient keyframe-based solutions are becoming the de facto methodology for building a monocular SLAM system. The objective of this paper is threefold: first, the paper serves as a guideline for people seeking to design their own monocular SLAM according to specific environmental constraints. Second, it presents a survey that covers the various keyframe-based monocular SLAM systems in the literature, detailing the components of their implementation, and critically assessing the specific strategies made in each proposed solution. Third, the paper provides insight into the direction of future research in this field, to address the major limitations still facing monocular SLAM; namely, in the issues of illumination changes, initialization, highly dynamic motion, poorly textured scenes, repetitive textures, map maintenance, and failure recovery.
Tasks
Published	2016-07-02
URL	http://arxiv.org/abs/1607.00470v2
PDF	http://arxiv.org/pdf/1607.00470v2.pdf
PWC	https://paperswithcode.com/paper/keyframe-based-monocular-slam-design-survey
Repo	https://github.com/adioshun/gitBook_DeepSlam
Framework	none

Deep Multi-task Representation Learning: A Tensor Factorisation Approach


Title	Deep Multi-task Representation Learning: A Tensor Factorisation Approach
Authors	Yongxin Yang, Timothy Hospedales
Abstract	Most contemporary multi-task learning methods assume linear models. This setting is considered shallow in the era of deep learning. In this paper, we present a new deep multi-task representation learning framework that learns cross-task sharing structure at every layer in a deep network. Our approach is based on generalising the matrix factorisation techniques explicitly or implicitly used by many conventional MTL algorithms to tensor factorisation, to realise automatic learning of end-to-end knowledge sharing in deep networks. This is in contrast to existing deep learning approaches that need a user-defined multi-task sharing strategy. Our approach applies to both homogeneous and heterogeneous MTL. Experiments demonstrate the efficacy of our deep multi-task representation learning in terms of both higher accuracy and fewer design choices.
Tasks	Multi-Task Learning, Representation Learning
Published	2016-05-20
URL	http://arxiv.org/abs/1605.06391v2
PDF	http://arxiv.org/pdf/1605.06391v2.pdf
PWC	https://paperswithcode.com/paper/deep-multi-task-representation-learning-a
Repo	https://github.com/safooray/tensor_factorization_mtl
Framework	tf

Convolutional Two-Stream Network Fusion for Video Action Recognition


Title	Convolutional Two-Stream Network Fusion for Video Action Recognition
Authors	Christoph Feichtenhofer, Axel Pinz, Andrew Zisserman
Abstract	Recent applications of Convolutional Neural Networks (ConvNets) for human action recognition in videos have proposed different solutions for incorporating the appearance and motion information. We study a number of ways of fusing ConvNet towers both spatially and temporally in order to best take advantage of this spatio-temporal information. We make the following findings: (i) that rather than fusing at the softmax layer, a spatial and temporal network can be fused at a convolution layer without loss of performance, but with a substantial saving in parameters; (ii) that it is better to fuse such networks spatially at the last convolutional layer than earlier, and that additionally fusing at the class prediction layer can boost accuracy; finally (iii) that pooling of abstract convolutional features over spatiotemporal neighbourhoods further boosts performance. Based on these studies we propose a new ConvNet architecture for spatiotemporal fusion of video snippets, and evaluate its performance on standard benchmarks where this architecture achieves state-of-the-art results.
Tasks	Action Recognition In Videos, Temporal Action Localization
Published	2016-04-22
URL	http://arxiv.org/abs/1604.06573v2
PDF	http://arxiv.org/pdf/1604.06573v2.pdf
PWC	https://paperswithcode.com/paper/convolutional-two-stream-network-fusion-for
Repo	https://github.com/feichtenhofer/twostreamfusion
Framework	none