October 20, 2019

3076 words 15 mins read

Paper Group ANR 68

Detailed Investigation of Deep Features with Sparse Representation and Dimensionality Reduction in CBIR: A Comparative Study. Pioneer Networks: Progressively Growing Generative Autoencoder. Piecewise Strong Convexity of Neural Networks. Gromov-Wasserstein Alignment of Word Embedding Spaces. Mathematical Analysis on Out-of-Sample Extensions. Active …

Detailed Investigation of Deep Features with Sparse Representation and Dimensionality Reduction in CBIR: A Comparative Study


Title	Detailed Investigation of Deep Features with Sparse Representation and Dimensionality Reduction in CBIR: A Comparative Study
Authors	Ahmad S. Tarawneh, Ceyhun Celik, Ahmad B. Hassanat, Dmitry Chetverikov
Abstract	Research on content-based image retrieval (CBIR) has been under development for decades, and numerous methods have been competing to extract the most discriminative features for improved representation of the image content. Recently, deep learning methods have gained attention in computer vision, including CBIR. In this paper, we present a comparative investigation of different features, including low-level and high-level features, for CBIR. We compare the performance of CBIR systems using different deep features with state-of-the-art low-level features such as SIFT, SURF, HOG, LBP, and LTP, using different dictionaries and coefficient learning techniques. Furthermore, we conduct comparisons with a set of primitive and popular features that have been used in this field, including colour histograms and Gabor features. We also investigate the discriminative power of deep features using certain similarity measures under different validation approaches. Furthermore, we investigate the effects of the dimensionality reduction of deep features on the performance of CBIR systems using principal component analysis, discrete wavelet transform, and discrete cosine transform. Unprecedentedly, the experimental results demonstrate high (95% and 93%) mean average precisions when using the VGG-16 FC7 deep features of Corel-1000 and Coil-20 datasets with 10-D and 20-D K-SVD, respectively.
Tasks	Content-Based Image Retrieval, Dimensionality Reduction, Image Retrieval
Published	2018-11-23
URL	http://arxiv.org/abs/1811.09681v1
PDF	http://arxiv.org/pdf/1811.09681v1.pdf
PWC	https://paperswithcode.com/paper/detailed-investigation-of-deep-features-with
Repo
Framework

Pioneer Networks: Progressively Growing Generative Autoencoder


Title	Pioneer Networks: Progressively Growing Generative Autoencoder
Authors	Ari Heljakka, Arno Solin, Juho Kannala
Abstract	We introduce a novel generative autoencoder network model that learns to encode and reconstruct images with high quality and resolution, and supports smooth random sampling from the latent space of the encoder. Generative adversarial networks (GANs) are known for their ability to simulate random high-quality images, but they cannot reconstruct existing images. Previous works have attempted to extend GANs to support such inference but, so far, have not delivered satisfactory high-quality results. Instead, we propose the Progressively Growing Generative Autoencoder (PIONEER) network which achieves high-quality reconstruction with $128{\times}128$ images without requiring a GAN discriminator. We merge recent techniques for progressively building up the parts of the network with the recently introduced adversarial encoder-generator network. The ability to reconstruct input images is crucial in many real-world applications, and allows for precise intelligent manipulation of existing images. We show promising results in image synthesis and inference, with state-of-the-art results in CelebA inference tasks.
Tasks	Image Generation
Published	2018-07-09
URL	http://arxiv.org/abs/1807.03026v2
PDF	http://arxiv.org/pdf/1807.03026v2.pdf
PWC	https://paperswithcode.com/paper/pioneer-networks-progressively-growing
Repo
Framework

Piecewise Strong Convexity of Neural Networks


Title	Piecewise Strong Convexity of Neural Networks
Authors	Tristan Milne
Abstract	We study the loss surface of a feed-forward neural network with ReLU non-linearities, regularized with weight decay. We show that the regularized loss function is piecewise strongly convex on an important open set which contains, under some conditions, all of its global minimizers. This is used to prove that local minima of the regularized loss function in this set are isolated, and that every differentiable critical point in this set is a local minimum, partially addressing an open problem given at the Conference on Learning Theory (COLT) 2015; our result is also applied to linear neural networks to show that with weight decay regularization, there are no non-zero critical points in a norm ball obtaining training error below a given threshold. We also include an experimental section where we validate our theoretical work and show that the regularized loss function is almost always piecewise strongly convex when restricted to stochastic gradient descent trajectories for three standard image classification problems.
Tasks	Image Classification
Published	2018-10-30
URL	https://arxiv.org/abs/1810.12805v3
PDF	https://arxiv.org/pdf/1810.12805v3.pdf
PWC	https://paperswithcode.com/paper/piecewise-strong-convexity-of-neural-networks
Repo
Framework

Gromov-Wasserstein Alignment of Word Embedding Spaces


Title	Gromov-Wasserstein Alignment of Word Embedding Spaces
Authors	David Alvarez-Melis, Tommi S. Jaakkola
Abstract	Cross-lingual or cross-domain correspondences play key roles in tasks ranging from machine translation to transfer learning. Recently, purely unsupervised methods operating on monolingual embeddings have become effective alignment tools. Current state-of-the-art methods, however, involve multiple steps, including heuristic post-hoc refinement strategies. In this paper, we cast the correspondence problem directly as an optimal transport (OT) problem, building on the idea that word embeddings arise from metric recovery algorithms. Indeed, we exploit the Gromov-Wasserstein distance that measures how similarities between pairs of words relate across languages. We show that our OT objective can be estimated efficiently, requires little or no tuning, and results in performance comparable with the state-of-the-art in various unsupervised word translation tasks.
Tasks	Machine Translation, Transfer Learning, Word Embeddings
Published	2018-08-31
URL	http://arxiv.org/abs/1809.00013v1
PDF	http://arxiv.org/pdf/1809.00013v1.pdf
PWC	https://paperswithcode.com/paper/gromov-wasserstein-alignment-of-word
Repo
Framework

Mathematical Analysis on Out-of-Sample Extensions


Title	Mathematical Analysis on Out-of-Sample Extensions
Authors	Jianzhong Wang
Abstract	Let $X=\mathbf{X}\cup\mathbf{Z}$ be a data set in $\mathbb{R}^D$, where $\mathbf{X}$ is the training set and $\mathbf{Z}$ is the test one. Many unsupervised learning algorithms based on kernel methods have been developed to provide dimensionality reduction (DR) embedding for a given training set $\Phi: \mathbf{X} \to \mathbb{R}^d$ ( $d\ll D$) that maps the high-dimensional data $\mathbf{X}$ to its low-dimensional feature representation $\mathbf{Y}=\Phi(\mathbf{X})$. However, these algorithms do not straightforwardly produce DR of the test set $\mathbf{Z}$. An out-of-sample extension method provides DR of $\mathbf{Z}$ using an extension of the existent embedding $\Phi$, instead of re-computing the DR embedding for the whole set $X$. Among various out-of-sample DR extension methods, those based on Nystr"{o}m approximation are very attractive. Many papers have developed such out-of-extension algorithms and shown their validity by numerical experiments. However, the mathematical theory for the DR extension still need further consideration. Utilizing the reproducing kernel Hilbert space (RKHS) theory, this paper develops a preliminary mathematical analysis on the out-of-sample DR extension operators. It treats an out-of-sample DR extension operator as an extension of the identity on the RKHS defined on $\mathbf{X}$. Then the Nystr"{o}m-type DR extension turns out to be an orthogonal projection. In the paper, we also present the conditions for the exact DR extension and give the estimate for the error of the extension.
Tasks	Dimensionality Reduction
Published	2018-04-19
URL	http://arxiv.org/abs/1804.09784v1
PDF	http://arxiv.org/pdf/1804.09784v1.pdf
PWC	https://paperswithcode.com/paper/mathematical-analysis-on-out-of-sample
Repo
Framework

Active query-driven visual search using probabilistic bisection and convolutional neural networks


Title	Active query-driven visual search using probabilistic bisection and convolutional neural networks
Authors	Athanasios Tsiligkaridis, Theodoros Tsiligkaridis
Abstract	We present a novel efficient object detection and localization framework based on the probabilistic bisection algorithm. A Convolutional Neural Network (CNN) is trained and used as a noisy oracle that provides answers to input query images. The responses along with error probability estimates obtained from the CNN are used to update beliefs on the object location along each dimension. We show that querying along each dimension achieves the same lower bound on localization error as the joint query design. Finally, we compare our approach to the traditional sliding window technique on a real world face localization task and show speed improvements by at least an order of magnitude while maintaining accurate localization.
Tasks	Object Detection
Published	2018-06-28
URL	https://arxiv.org/abs/1806.11223v3
PDF	https://arxiv.org/pdf/1806.11223v3.pdf
PWC	https://paperswithcode.com/paper/active-query-driven-visual-search-using
Repo
Framework

Superhighway: Bypass Data Sparsity in Cross-Domain CF


Title	Superhighway: Bypass Data Sparsity in Cross-Domain CF
Authors	Kwei-Herng Lai, Ting-Hsiang Wang, Heng-Yu Chi, Yian Chen, Ming-Feng Tsai, Chuan-Ju Wang
Abstract	Cross-domain collaborative filtering (CF) aims to alleviate data sparsity in single-domain CF by leveraging knowledge transferred from related domains. Many traditional methods focus on enriching compared neighborhood relations in CF directly to address the sparsity problem. In this paper, we propose superhighway construction, an alternative explicit relation-enrichment procedure, to improve recommendations by enhancing cross-domain connectivity. Specifically, assuming partially overlapped items (users), superhighway bypasses multi-hop inter-domain paths between cross-domain users (items, respectively) with direct paths to enrich the cross-domain connectivity. The experiments conducted on a real-world cross-region music dataset and a cross-platform movie dataset show that the proposed superhighway construction significantly improves recommendation performance in both target and source domains.
Tasks
Published	2018-08-28
URL	http://arxiv.org/abs/1808.09784v1
PDF	http://arxiv.org/pdf/1808.09784v1.pdf
PWC	https://paperswithcode.com/paper/superhighway-bypass-data-sparsity-in-cross
Repo
Framework

The Vertex Sample Complexity of Free Energy is Polynomial


Title	The Vertex Sample Complexity of Free Energy is Polynomial
Authors	Vishesh Jain, Frederic Koehler, Elchanan Mossel
Abstract	We study the following question: given a massive Markov random field on $n$ nodes, can a small sample from it provide a rough approximation to the free energy $\mathcal{F}_n = \log{Z_n}$? Results in graph limit literature by Borgs, Chayes, Lov'asz, S'os, and Vesztergombi show that for Ising models on $n$ nodes and interactions of strength $\Theta(1/n)$, an $\epsilon$ approximation to $\log Z_n / n$ can be achieved by sampling a randomly induced model on $2^{O(1/\epsilon^2)}$ nodes. We show that the sampling complexity of this problem is {\em polynomial in} $1/\epsilon$. We further show a polynomial dependence on $\epsilon$ cannot be avoided. Our results are very general as they apply to higher order Markov random fields. For Markov random fields of order $r$, we obtain an algorithm that achieves $\epsilon$ approximation using a number of samples polynomial in $r$ and $1/\epsilon$ and running time that is $2^{O(1/\epsilon^2)}$ up to polynomial factors in $r$ and $\epsilon$. For ferromagnetic Ising models, the running time is polynomial in $1/\epsilon$. Our results are intimately connected to recent research on the regularity lemma and property testing, where the interest is in finding which properties can tested within $\epsilon$ error in time polynomial in $1/\epsilon$. In particular, our proofs build on results from a recent work by Alon, de la Vega, Kannan and Karpinski, who also introduced the notion of polynomial vertex sample complexity. Another critical ingredient of the proof is an effective bound by the authors of the paper relating the variational free energy and the free energy.
Tasks
Published	2018-02-16
URL	http://arxiv.org/abs/1802.06129v2
PDF	http://arxiv.org/pdf/1802.06129v2.pdf
PWC	https://paperswithcode.com/paper/the-vertex-sample-complexity-of-free-energy
Repo
Framework

The ADAPT System Description for the IWSLT 2018 Basque to English Translation Task


Title	The ADAPT System Description for the IWSLT 2018 Basque to English Translation Task
Authors	Alberto Poncelas, Andy Way, Kepa Sarasola
Abstract	In this paper we present the ADAPT system built for the Basque to English Low Resource MT Evaluation Campaign. Basque is a low-resourced, morphologically-rich language. This poses a challenge for Neural Machine Translation models which usually achieve better performance when trained with large sets of data. Accordingly, we used synthetic data to improve the translation quality produced by a model built using only authentic data. Our proposal uses back-translated data to: (a) create new sentences, so the system can be trained with more data; and (b) translate sentences that are close to the test set, so the model can be fine-tuned to the document to be translated.
Tasks	Machine Translation
Published	2018-11-14
URL	http://arxiv.org/abs/1811.05909v1
PDF	http://arxiv.org/pdf/1811.05909v1.pdf
PWC	https://paperswithcode.com/paper/the-adapt-system-description-for-the-iwslt
Repo
Framework

CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving


Title	CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving
Authors	Xiaodan Liang, Tairui Wang, Luona Yang, Eric Xing
Abstract	Autonomous urban driving navigation with complex multi-agent dynamics is under-explored due to the difficulty of learning an optimal driving policy. The traditional modular pipeline heavily relies on hand-designed rules and the pre-processing perception system while the supervised learning-based models are limited by the accessibility of extensive human experience. We present a general and principled Controllable Imitative Reinforcement Learning (CIRL) approach which successfully makes the driving agent achieve higher success rates based on only vision inputs in a high-fidelity car simulator. To alleviate the low exploration efficiency for large continuous action space that often prohibits the use of classical RL on challenging real tasks, our CIRL explores over a reasonably constrained action space guided by encoded experiences that imitate human demonstrations, building upon Deep Deterministic Policy Gradient (DDPG). Moreover, we propose to specialize adaptive policies and steering-angle reward designs for different control signals (i.e. follow, straight, turn right, turn left) based on the shared representations to improve the model capability in tackling with diverse cases. Extensive experiments on CARLA driving benchmark demonstrate that CIRL substantially outperforms all previous methods in terms of the percentage of successfully completed episodes on a variety of goal-directed driving tasks. We also show its superior generalization capability in unseen environments. To our knowledge, this is the first successful case of the learned driving policy through reinforcement learning in the high-fidelity simulator, which performs better-than supervised imitation learning.
Tasks	Imitation Learning
Published	2018-07-10
URL	http://arxiv.org/abs/1807.03776v1
PDF	http://arxiv.org/pdf/1807.03776v1.pdf
PWC	https://paperswithcode.com/paper/cirl-controllable-imitative-reinforcement
Repo
Framework

Training Millions of Personalized Dialogue Agents


Title	Training Millions of Personalized Dialogue Agents
Authors	Pierre-Emmanuel Mazaré, Samuel Humeau, Martin Raison, Antoine Bordes
Abstract	Current dialogue systems are not very engaging for users, especially when trained end-to-end without relying on proactive reengaging scripted strategies. Zhang et al. (2018) showed that the engagement level of end-to-end dialogue models increases when conditioning them on text personas providing some personalized back-story to the model. However, the dataset used in Zhang et al. (2018) is synthetic and of limited size as it contains around 1k different personas. In this paper we introduce a new dataset providing 5 million personas and 700 million persona-based dialogues. Our experiments show that, at this scale, training using personas still improves the performance of end-to-end systems. In addition, we show that other tasks benefit from the wide coverage of our dataset by fine-tuning our model on the data from Zhang et al. (2018) and achieving state-of-the-art results.
Tasks
Published	2018-09-06
URL	http://arxiv.org/abs/1809.01984v1
PDF	http://arxiv.org/pdf/1809.01984v1.pdf
PWC	https://paperswithcode.com/paper/training-millions-of-personalized-dialogue
Repo
Framework

Analysis of cause-effect inference by comparing regression errors


Title	Analysis of cause-effect inference by comparing regression errors
Authors	Patrick Blöbaum, Dominik Janzing, Takashi Washio, Shohei Shimizu, Bernhard Schölkopf
Abstract	We address the problem of inferring the causal direction between two variables by comparing the least-squares errors of the predictions in both possible directions. Under the assumption of an independence between the function relating cause and effect, the conditional noise distribution, and the distribution of the cause, we show that the errors are smaller in causal direction if both variables are equally scaled and the causal relation is close to deterministic. Based on this, we provide an easily applicable algorithm that only requires a regression in both possible causal directions and a comparison of the errors. The performance of the algorithm is compared with various related causal inference methods in different artificial and real-world data sets.
Tasks	Causal Inference
Published	2018-02-19
URL	http://arxiv.org/abs/1802.06698v2
PDF	http://arxiv.org/pdf/1802.06698v2.pdf
PWC	https://paperswithcode.com/paper/analysis-of-cause-effect-inference-by
Repo
Framework

Local Distribution in Neighborhood for Classification


Title	Local Distribution in Neighborhood for Classification
Authors	Chengsheng Mao, Bin Hu, Lei Chen, Philip Moore, Xiaowei Zhang
Abstract	The k-nearest-neighbor method performs classification tasks for a query sample based on the information contained in its neighborhood. Previous studies into the k-nearest-neighbor algorithm usually achieved the decision value for a class by combining the support of each sample in the neighborhood. They have generally considered the nearest neighbors separately, and potentially integral neighborhood information important for classification was lost, e.g. the distribution information. This article proposes a novel local learning method that organizes the information in the neighborhood through local distribution. In the proposed method, additional distribution information in the neighborhood is estimated and then organized; the classification decision is made based on maximum posterior probability which is estimated from the local distribution in the neighborhood. Additionally, based on the local distribution, we generate a generalized local classification form that can be effectively applied to various datasets through tuning the parameters. We use both synthetic and real datasets to evaluate the classification performance of the proposed method; the experimental results demonstrate the dimensional scalability, efficiency, effectiveness and robustness of the proposed method compared to some other state-of-the-art classifiers. The results indicate that the proposed method is effective and promising in a broad range of domains.
Tasks
Published	2018-12-07
URL	http://arxiv.org/abs/1812.02934v1
PDF	http://arxiv.org/pdf/1812.02934v1.pdf
PWC	https://paperswithcode.com/paper/local-distribution-in-neighborhood-for
Repo
Framework

Attention-based Active Visual Search for Mobile Robots


Title	Attention-based Active Visual Search for Mobile Robots
Authors	Amir Rasouli, Pablo Lanillos, Gordon Cheng, John K. Tsotsos
Abstract	We present an active visual search model for finding objects in unknown environments. The proposed algorithm guides the robot towards the sought object using the relevant stimuli provided by the visual sensors. Existing search strategies are either purely reactive or use simplified sensor models that do not exploit all the visual information available. In this paper, we propose a new model that actively extracts visual information via visual attention techniques and, in conjunction with a non-myopic decision-making algorithm, leads the robot to search more relevant areas of the environment. The attention module couples both top-down and bottom-up attention models enabling the robot to search regions with higher importance first. The proposed algorithm is evaluated on a mobile robot platform in a 3D simulated environment. The results indicate that the use of visual attention significantly improves search, but the degree of improvement depends on the nature of the task and the complexity of the environment. In our experiments, we found that performance enhancements of up to 42% in structured and 38% in highly unstructured cluttered environments can be achieved using visual attention mechanisms.
Tasks	Decision Making
Published	2018-07-27
URL	http://arxiv.org/abs/1807.10744v1
PDF	http://arxiv.org/pdf/1807.10744v1.pdf
PWC	https://paperswithcode.com/paper/attention-based-active-visual-search-for
Repo
Framework

GrCAN: Gradient Boost Convolutional Autoencoder with Neural Decision Forest


Title	GrCAN: Gradient Boost Convolutional Autoencoder with Neural Decision Forest
Authors	Manqing Dong, Lina Yao, Xianzhi Wang, Boualem Benatallah, Shuai Zhang
Abstract	Random forest and deep neural network are two schools of effective classification methods in machine learning. While the random forest is robust irrespective of the data domain, the deep neural network has advantages in handling high dimensional data. In view that a differentiable neural decision forest can be added to the neural network to fully exploit the benefits of both models, in our work, we further combine convolutional autoencoder with neural decision forest, where autoencoder has its advantages in finding the hidden representations of the input data. We develop a gradient boost module and embed it into the proposed convolutional autoencoder with neural decision forest to improve the performance. The idea of gradient boost is to learn and use the residual in the prediction. In addition, we design a structure to learn the parameters of the neural decision forest and gradient boost module at contiguous steps. The extensive experiments on several public datasets demonstrate that our proposed model achieves good efficiency and prediction performance compared with a series of baseline methods.
Tasks
Published	2018-06-21
URL	http://arxiv.org/abs/1806.08079v2
PDF	http://arxiv.org/pdf/1806.08079v2.pdf
PWC	https://paperswithcode.com/paper/grcan-gradient-boost-convolutional
Repo
Framework