October 19, 2019

3205 words 16 mins read

Paper Group ANR 346

Totally Looks Like - How Humans Compare, Compared to Machines. Stein Variational Gradient Descent Without Gradient. On sampling from a log-concave density using kinetic Langevin diffusions. Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis. Text Classification using Capsules. Connecting …

Totally Looks Like - How Humans Compare, Compared to Machines


Title	Totally Looks Like - How Humans Compare, Compared to Machines
Authors	Amir Rosenfeld, Markus D. Solbach, John K. Tsotsos
Abstract	Perceptual judgment of image similarity by humans relies on rich internal representations ranging from low-level features to high-level concepts, scene properties and even cultural associations. However, existing methods and datasets attempting to explain perceived similarity use stimuli which arguably do not cover the full breadth of factors that affect human similarity judgments, even those geared toward this goal. We introduce a new dataset dubbed Totally-Looks-Like (TLL) after a popular entertainment website, which contains images paired by humans as being visually similar. The dataset contains 6016 image-pairs from the wild, shedding light upon a rich and diverse set of criteria employed by human beings. We conduct experiments to try to reproduce the pairings via features extracted from state-of-the-art deep convolutional neural networks, as well as additional human experiments to verify the consistency of the collected data. Though we create conditions to artificially make the matching task increasingly easier, we show that machine-extracted representations perform very poorly in terms of reproducing the matching selected by humans. We discuss and analyze these results, suggesting future directions for improvement of learned image representations.
Tasks
Published	2018-03-05
URL	http://arxiv.org/abs/1803.01485v3
PDF	http://arxiv.org/pdf/1803.01485v3.pdf
PWC	https://paperswithcode.com/paper/totally-looks-like-how-humans-compare
Repo
Framework

Stein Variational Gradient Descent Without Gradient


Title	Stein Variational Gradient Descent Without Gradient
Authors	Jun Han, Qiang Liu
Abstract	Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for complex distributions. However, the standard SVGD requires calculating the gradient of the target density and cannot be applied when the gradient is unavailable. In this work, we develop a gradient-free variant of SVGD (GF-SVGD), which replaces the true gradient with a surrogate gradient, and corrects the induced bias by re-weighting the gradients in a proper form. We show that our GF-SVGD can be viewed as the standard SVGD with a special choice of kernel, and hence directly inherits the theoretical properties of SVGD. We shed insights on the empirical choice of the surrogate gradient and propose an annealed GF-SVGD that leverages the idea of simulated annealing to improve the performance on high dimensional complex distributions. Empirical studies show that our method consistently outperforms a number of recent advanced gradient-free MCMC methods.
Tasks
Published	2018-06-07
URL	http://arxiv.org/abs/1806.02775v1
PDF	http://arxiv.org/pdf/1806.02775v1.pdf
PWC	https://paperswithcode.com/paper/stein-variational-gradient-descent-without
Repo
Framework

On sampling from a log-concave density using kinetic Langevin diffusions


Title	On sampling from a log-concave density using kinetic Langevin diffusions
Authors	Arnak S. Dalalyan, Lionel Riou-Durand
Abstract	Langevin diffusion processes and their discretizations are often used for sampling from a target density. The most convenient framework for assessing the quality of such a sampling scheme corresponds to smooth and strongly log-concave densities defined on $\mathbb R^p$. The present work focuses on this framework and studies the behavior of Monte Carlo algorithms based on discretizations of the kinetic Langevin diffusion. We first prove the geometric mixing property of the kinetic Langevin diffusion with a mixing rate that is, in the overdamped regime, optimal in terms of its dependence on the condition number. We then use this result for obtaining improved guarantees of sampling using the kinetic Langevin Monte Carlo method, when the quality of sampling is measured by the Wasserstein distance. We also consider the situation where the Hessian of the log-density of the target distribution is Lipschitz-continuous. In this case, we introduce a new discretization of the kinetic Langevin diffusion and prove that this leads to a substantial improvement of the upper bound on the sampling error measured in Wasserstein distance.
Tasks
Published	2018-07-24
URL	http://arxiv.org/abs/1807.09382v6
PDF	http://arxiv.org/pdf/1807.09382v6.pdf
PWC	https://paperswithcode.com/paper/on-sampling-from-a-log-concave-density-using
Repo
Framework

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis


Title	Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis
Authors	Kelly W. Zhang, Samuel R. Bowman
Abstract	Recent work using auxiliary prediction task classifiers to investigate the properties of LSTM representations has begun to shed light on why pretrained representations, like ELMo (Peters et al., 2018) and CoVe (McCann et al., 2017), are so beneficial for neural language understanding models. We still, though, do not yet have a clear understanding of how the choice of pretraining objective affects the type of linguistic information that models learn. With this in mind, we compare four objectives—language modeling, translation, skip-thought, and autoencoding—on their ability to induce syntactic and part-of-speech information. We make a fair comparison between the tasks by holding constant the quantity and genre of the training data, as well as the LSTM architecture. We find that representations from language models consistently perform best on our syntactic auxiliary prediction tasks, even when trained on relatively small amounts of data. These results suggest that language modeling may be the best data-rich pretraining task for transfer learning applications requiring syntactic information. We also find that the representations from randomly-initialized, frozen LSTMs perform strikingly well on our syntactic auxiliary tasks, but this effect disappears when the amount of training data for the auxiliary tasks is reduced.
Tasks	Language Modelling, Transfer Learning
Published	2018-09-26
URL	http://arxiv.org/abs/1809.10040v2
PDF	http://arxiv.org/pdf/1809.10040v2.pdf
PWC	https://paperswithcode.com/paper/language-modeling-teaches-you-more-syntax
Repo
Framework

Text Classification using Capsules


Title	Text Classification using Capsules
Authors	Jaeyoung Kim, Sion Jang, Sungchul Choi, Eunjeong Park
Abstract	This paper presents an empirical exploration of the use of capsule networks for text classification. While it has been shown that capsule networks are effective for image classification, their validity in the domain of text has not been explored. In this paper, we show that capsule networks indeed have the potential for text classification and that they have several advantages over convolutional neural networks. We further suggest a simple routing method that effectively reduces the computational complexity of dynamic routing. We utilized seven benchmark datasets to demonstrate that capsule networks, along with the proposed routing method provide comparable results.
Tasks	Image Classification, Text Classification
Published	2018-08-12
URL	http://arxiv.org/abs/1808.03976v2
PDF	http://arxiv.org/pdf/1808.03976v2.pdf
PWC	https://paperswithcode.com/paper/text-classification-using-capsules
Repo
Framework

Connecting Spectral Clustering to Maximum Margins and Level Sets


Title	Connecting Spectral Clustering to Maximum Margins and Level Sets
Authors	David P. Hofmeyr
Abstract	We study the connections between spectral clustering and the problems of maximum margin clustering, and estimation of the components of level sets of a density function. Specifically, we obtain bounds on the eigenvectors of graph Laplacian matrices in terms of the between cluster separation, and within cluster connectivity. These bounds ensure that the spectral clustering solution converges to the maximum margin clustering solution as the scaling parameter is reduced towards zero. The sensitivity of maximum margin clustering solutions to outlying points is well known, but can be mitigated by first removing such outliers, and applying maximum margin clustering to the remaining points. If outliers are identified using an estimate of the underlying probability density, then the remaining points may be seen as an estimate of a level set of this density function. We show that such an approach can be used to consistently estimate the components of the level sets of a density function under very mild assumptions.
Tasks
Published	2018-12-16
URL	http://arxiv.org/abs/1812.06397v1
PDF	http://arxiv.org/pdf/1812.06397v1.pdf
PWC	https://paperswithcode.com/paper/connecting-spectral-clustering-to-maximum
Repo
Framework

Theory and Algorithms for Pulse Signal Processing


Title	Theory and Algorithms for Pulse Signal Processing
Authors	Gabriel Nallathambi, Jose C. Principe
Abstract	The integrate and fire converter transforms an analog signal into train of biphasic pulses. The pulse train has information encoded in the timing and polarity of pulses. While it has been shown that any finite bandwidth analog signal can be reconstructed from these pulse trains with an error as small as desired, there is a need for fundamental signal processing techniques to operate directly on pulse trains without signal reconstruction. In this paper, the feasibility of performing online the signal processing operations of addition, multiplication, and convolution of analog signals using their pulses train representations is explored. Theoretical framework to perform signal processing with pulse trains imposing minimal restrictions is derived, and algorithms for online implementation of the operators are developed. Performance of the algorithms in processing simulated data is studied. An application of noise subtraction and representation of relevant features of interest in electrocardiogram signal is demonstrated with mean pulse rate less than 20 pulses per second.
Tasks
Published	2018-12-31
URL	http://arxiv.org/abs/1901.01140v1
PDF	http://arxiv.org/pdf/1901.01140v1.pdf
PWC	https://paperswithcode.com/paper/theory-and-algorithms-for-pulse-signal
Repo
Framework

Using External Archive for Improved Performance in Multi-Objective Optimization


Title	Using External Archive for Improved Performance in Multi-Objective Optimization
Authors	Mahesh B. Patil
Abstract	It is shown that the use of an external archive, purely for storage purposes, can bring substantial benefits in multi-objective optimization. A new scheme for archive management for the above purpose is described. The new scheme is combined with the NSGA-II algorithm for solving two multi-objective optimization problems, and it is demonstrated that this combination gives significantly improved sets of Pareto-optimal solutions. The additional computational effort because of the external archive is found to be insignificant when the objective functions are expensive to evaluate.
Tasks
Published	2018-11-22
URL	http://arxiv.org/abs/1811.09196v1
PDF	http://arxiv.org/pdf/1811.09196v1.pdf
PWC	https://paperswithcode.com/paper/using-external-archive-for-improved
Repo
Framework

DEEPGONET: Multi-label Prediction of GO Annotation for Protein from Sequence Using Cascaded Convolutional and Recurrent Network


Title	DEEPGONET: Multi-label Prediction of GO Annotation for Protein from Sequence Using Cascaded Convolutional and Recurrent Network
Authors	Sheikh Muhammad Saiful Islam, Md Mahedi Hasan
Abstract	The present gap between the amount of available protein sequence due to the development of next generation sequencing technology (NGS) and slow and expensive experimental extraction of useful information like annotation of protein sequence in different functional aspects, is ever widening, which can be reduced by employing automatic function prediction (AFP) approaches. Gene Ontology (GO), comprising of more than 40, 000 classes, defines three aspects of protein function names Biological Process (BP), Cellular Component (CC), Molecular Function (MF). Multiple functions of a single protein, has made automatic function prediction a large-scale, multi-class, multi-label task. In this paper, we present DEEPGONET, a novel cascaded convolutional and recurrent neural network, to predict the top-level hierarchy of GO ontology. The network takes the primary sequence of protein as input which makes it more useful than other prevailing state-of-the-art deep learning based methods with multi-modal input, making them less applicable for proteins where only primary sequence is available. All predictions of different protein functions of our network are performed by the same architecture, a proof of better generalization as demonstrated by promising performance on a variety of organisms while trained on Homo sapiens only, which is made possible by efficient exploration of vast output space by leveraging hierarchical relationship among GO classes. The promising performance of our model makes it a potential avenue for directing experimental protein functions exploration efficiently by vastly eliminating possible routes which is done by the exploring only the suggested routes from our model. Our proposed model is also very simple and efficient in terms of computational time and space compared to other architectures in literature.
Tasks	Efficient Exploration
Published	2018-10-31
URL	http://arxiv.org/abs/1811.00053v1
PDF	http://arxiv.org/pdf/1811.00053v1.pdf
PWC	https://paperswithcode.com/paper/deepgonet-multi-label-prediction-of-go
Repo
Framework

A Hierarchical Matcher using Local Classifier Chains


Title	A Hierarchical Matcher using Local Classifier Chains
Authors	Lingfeng Zhang, Ioannis A. Kakadiaris
Abstract	This paper focuses on improving the performance of current convolutional neural networks in visual recognition without changing the network architecture. A hierarchical matcher is proposed that builds chains of local binary neural networks after one global neural network over all the class labels, named as Local Classifier Chains based Convolutional Neural Network (LCC-CNN). The signature of each sample as two components: global component based on the global network; local component based on local binary networks. The local networks are built based on label pairs created by a similarity matrix and confusion matrix. During matching, each sample travels through one global network and a chain of local networks to obtain its final matching to avoid error propagation. The proposed matcher has been evaluated with image recognition, character recognition and face recognition datasets. The experimental results indicate that the proposed matcher achieves better performance when compared with methods using only a global deep network. Compared with the UR2D system, the accuracy is improved significantly by 1% and 0.17% on the UHDB31 dataset and the IJB-A dataset, respectively.
Tasks	Face Recognition
Published	2018-05-07
URL	http://arxiv.org/abs/1805.02339v1
PDF	http://arxiv.org/pdf/1805.02339v1.pdf
PWC	https://paperswithcode.com/paper/a-hierarchical-matcher-using-local-classifier
Repo
Framework

Impact of Data Normalization on Deep Neural Network for Time Series Forecasting


Title	Impact of Data Normalization on Deep Neural Network for Time Series Forecasting
Authors	Samit Bhanja, Abhishek Das
Abstract	For the last few years it has been observed that the Deep Neural Networks (DNNs) has achieved an excellent success in image classification, speech recognition. But DNNs are suffer great deal of challenges for time series forecasting because most of the time series data are nonlinear in nature and highly dynamic in behaviour. The time series forecasting has a great impact on our socio-economic environment. Hence, to deal with these challenges its need to be redefined the DNN model and keeping this in mind, data pre-processing, network architecture and network parameters are need to be consider before feeding the data into DNN models. Data normalization is the basic data pre-processing technique form which learning is to be done. The effectiveness of time series forecasting is heavily depend on the data normalization technique. In this paper, different normalization methods are used on time series data before feeding the data into the DNN model and we try to find out the impact of each normalization technique on DNN to forecast the time series. Here the Deep Recurrent Neural Network (DRNN) is used to predict the closing index of Bombay Stock Exchange (BSE) and New York Stock Exchange (NYSE) by using BSE and NYSE time series data.
Tasks	Image Classification, Speech Recognition, Time Series, Time Series Forecasting
Published	2018-12-13
URL	http://arxiv.org/abs/1812.05519v2
PDF	http://arxiv.org/pdf/1812.05519v2.pdf
PWC	https://paperswithcode.com/paper/impact-of-data-normalization-on-deep-neural
Repo
Framework

An Interpretable Deep Hierarchical Semantic Convolutional Neural Network for Lung Nodule Malignancy Classification


Title	An Interpretable Deep Hierarchical Semantic Convolutional Neural Network for Lung Nodule Malignancy Classification
Authors	Shiwen Shen, Simon X. Han, Denise R. Aberle, Alex A. T. Bui, Willliam Hsu
Abstract	While deep learning methods are increasingly being applied to tasks such as computer-aided diagnosis, these models are difficult to interpret, do not incorporate prior domain knowledge, and are often considered as a “black-box.” The lack of model interpretability hinders them from being fully understood by target users such as radiologists. In this paper, we present a novel interpretable deep hierarchical semantic convolutional neural network (HSCNN) to predict whether a given pulmonary nodule observed on a computed tomography (CT) scan is malignant. Our network provides two levels of output: 1) low-level radiologist semantic features, and 2) a high-level malignancy prediction score. The low-level semantic outputs quantify the diagnostic features used by radiologists and serve to explain how the model interprets the images in an expert-driven manner. The information from these low-level tasks, along with the representations learned by the convolutional layers, are then combined and used to infer the high-level task of predicting nodule malignancy. This unified architecture is trained by optimizing a global loss function including both low- and high-level tasks, thereby learning all the parameters within a joint framework. Our experimental results using the Lung Image Database Consortium (LIDC) show that the proposed method not only produces interpretable lung cancer predictions but also achieves significantly better results compared to common 3D CNN approaches.
Tasks	Computed Tomography (CT)
Published	2018-06-02
URL	http://arxiv.org/abs/1806.00712v1
PDF	http://arxiv.org/pdf/1806.00712v1.pdf
PWC	https://paperswithcode.com/paper/an-interpretable-deep-hierarchical-semantic
Repo
Framework

Multivariate Forecasting of Crude Oil Spot Prices using Neural Networks


Title	Multivariate Forecasting of Crude Oil Spot Prices using Neural Networks
Authors	Ganapathy S. Natarajan, Aishwarya Ashok
Abstract	Crude oil is a major component in most advanced economies of the world. Accurately predicting and understanding the behavior of crude oil prices is important for economists, analysts, forecasters, and traders, to name a few. The price of crude oil has declined in the past decade and is seeing a phase of stability; but will this stability last? This work is an empirical study on how multivariate analysis may be employed to predict crude oil spot prices using neural networks. The concept of using neural networks showed promising potential. A very simple neural network model was able to perform on par with ARIMA models - the state-of-the-art model in time-series forecasting. Advanced neural network models using larger datasets may be used in the future to extend this proof-of-concept to a full scale framework.
Tasks	Time Series, Time Series Forecasting
Published	2018-11-21
URL	http://arxiv.org/abs/1811.08963v1
PDF	http://arxiv.org/pdf/1811.08963v1.pdf
PWC	https://paperswithcode.com/paper/multivariate-forecasting-of-crude-oil-spot
Repo
Framework

Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation


Title	Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation
Authors	Ali Can Kocabiyikoglu, Laurent Besacier, Olivier Kraif
Abstract	Recent works in spoken language translation (SLT) have attempted to build end-to-end speech-to-text translation without using source language transcription during learning or decoding. However, while large quantities of parallel texts (such as Europarl, OpenSubtitles) are available for training machine translation systems, there are no large (100h) and open source parallel corpora that include speech in a source language aligned to text in a target language. This paper tries to fill this gap by augmenting an existing (monolingual) corpus: LibriSpeech. This corpus, used for automatic speech recognition, is derived from read audiobooks from the LibriVox project, and has been carefully segmented and aligned. After gathering French e-books corresponding to the English audio-books from LibriSpeech, we align speech segments at the sentence level with their respective translations and obtain 236h of usable parallel data. This paper presents the details of the processing as well as a manual evaluation conducted on a small subset of the corpus. This evaluation shows that the automatic alignments scores are reasonably correlated with the human judgments of the bilingual alignment quality. We believe that this corpus (which is made available online) is useful for replicable experiments in direct speech translation or more general spoken language translation experiments.
Tasks	Machine Translation, Speech Recognition
Published	2018-02-09
URL	http://arxiv.org/abs/1802.03142v1
PDF	http://arxiv.org/pdf/1802.03142v1.pdf
PWC	https://paperswithcode.com/paper/augmenting-librispeech-with-french
Repo
Framework

Dr. Tux: A Question Answering System for Ubuntu users


Title	Dr. Tux: A Question Answering System for Ubuntu users
Authors	Bijil Abraham Philip, Manas Jog, Apurv Milind Upasani
Abstract	Various forums and question answering (Q&A) sites are available online that allow Ubuntu users to find results similar to their queries. However, searching for a result is often time consuming as it requires the user to find a specific problem instance relevant to his/her query from a large set of questions. In this paper, we present an automated question answering system for Ubuntu users called Dr. Tux that is designed to answer user’s queries by selecting the most similar question from an online database. The prototype was implemented in Python and uses NLTK and CoreNLP tools for Natural Language Processing. The data for the prototype was taken from the AskUbuntu website which contains about 150k questions. The results obtained from the manual evaluation of the prototype were promising while also presenting some interesting opportunities for improvement.
Tasks	Question Answering
Published	2018-08-25
URL	http://arxiv.org/abs/1808.08357v1
PDF	http://arxiv.org/pdf/1808.08357v1.pdf
PWC	https://paperswithcode.com/paper/dr-tux-a-question-answering-system-for-ubuntu
Repo
Framework