May 6, 2019

2835 words 14 mins read

Paper Group ANR 369

Neural Machine Translation with Supervised Attention. Channel Equalization Using Multilayer Perceptron Networks. On Some Properties of Calibrated Trifocal Tensors. Efficient Summarization with Read-Again and Copy Mechanism. A Step from Probabilistic Programming to Cognitive Architectures. Complexity of Discrete Energy Minimization Problems. Learnin …

Neural Machine Translation with Supervised Attention


Title	Neural Machine Translation with Supervised Attention
Authors	Lemao Liu, Masao Utiyama, Andrew Finch, Eiichiro Sumita
Abstract	The attention mechanisim is appealing for neural machine translation, since it is able to dynam- ically encode a source sentence by generating a alignment between a target word and source words. Unfortunately, it has been proved to be worse than conventional alignment models in aligment accuracy. In this paper, we analyze and explain this issue from the point view of re- ordering, and propose a supervised attention which is learned with guidance from conventional alignment models. Experiments on two Chinese-to-English translation tasks show that the super- vised attention mechanism yields better alignments leading to substantial gains over the standard attention based NMT.
Tasks	Machine Translation
Published	2016-09-14
URL	http://arxiv.org/abs/1609.04186v1
PDF	http://arxiv.org/pdf/1609.04186v1.pdf
PWC	https://paperswithcode.com/paper/neural-machine-translation-with-supervised
Repo
Framework

Channel Equalization Using Multilayer Perceptron Networks


Title	Channel Equalization Using Multilayer Perceptron Networks
Authors	Saba Baloch, Javed Ali Baloch, Mukhtiar Ali Unar
Abstract	In most digital communication systems, bandwidth limited channel along with multipath propagation causes ISI (Inter Symbol Interference) to occur. This phenomenon causes distortion of the given transmitted symbol due to other transmitted symbols. With the help of equalization ISI can be reduced. This paper presents a solution to the ISI problem by performing blind equalization using ANN (Artificial Neural Networks). The simulated network is a multilayer feedforward Perceptron ANN, which has been trained by utilizing the error back-propagation algorithm. The weights of the network are updated in accordance with training of the network. This paper presents a very effective method for blind channel equalization, being more efficient than the pre-existing algorithms. The obtained results show a visible reduction in the noise content.
Tasks
Published	2016-04-02
URL	http://arxiv.org/abs/1604.00558v1
PDF	http://arxiv.org/pdf/1604.00558v1.pdf
PWC	https://paperswithcode.com/paper/channel-equalization-using-multilayer
Repo
Framework

On Some Properties of Calibrated Trifocal Tensors


Title	On Some Properties of Calibrated Trifocal Tensors
Authors	Evgeniy Martyushev
Abstract	In two-view geometry, the essential matrix describes the relative position and orientation of two calibrated images. In three views, a similar role is assigned to the calibrated trifocal tensor. It is a particular case of the (uncalibrated) trifocal tensor and thus it inherits all its properties but, due to the smaller degrees of freedom, satisfies a number of additional algebraic constraints. Some of them are described in this paper. More specifically, we define a new notion — the trifocal essential matrix. On the one hand, it is a generalization of the ordinary (bifocal) essential matrix, and, on the other hand, it is closely related to the calibrated trifocal tensor. We prove the two necessary and sufficient conditions that characterize the set of trifocal essential matrices. Based on these characterizations, we propose three necessary conditions on a calibrated trifocal tensor. They have a form of 15 quartic and 99 quintic polynomial equations. We show that in the practically significant real case the 15 quartic constraints are also sufficient.
Tasks
Published	2016-01-07
URL	http://arxiv.org/abs/1601.01467v3
PDF	http://arxiv.org/pdf/1601.01467v3.pdf
PWC	https://paperswithcode.com/paper/on-some-properties-of-calibrated-trifocal
Repo
Framework

Efficient Summarization with Read-Again and Copy Mechanism


Title	Efficient Summarization with Read-Again and Copy Mechanism
Authors	Wenyuan Zeng, Wenjie Luo, Sanja Fidler, Raquel Urtasun
Abstract	Encoder-decoder models have been widely used to solve sequence to sequence prediction tasks. However current approaches suffer from two shortcomings. First, the encoders compute a representation of each word taking into account only the history of the words it has read so far, yielding suboptimal representations. Second, current decoders utilize large vocabularies in order to minimize the problem of unknown words, resulting in slow decoding times. In this paper we address both shortcomings. Towards this goal, we first introduce a simple mechanism that first reads the input sequence before committing to a representation of each word. Furthermore, we propose a simple copy mechanism that is able to exploit very small vocabularies and handle out-of-vocabulary words. We demonstrate the effectiveness of our approach on the Gigaword dataset and DUC competition outperforming the state-of-the-art.
Tasks
Published	2016-11-10
URL	http://arxiv.org/abs/1611.03382v1
PDF	http://arxiv.org/pdf/1611.03382v1.pdf
PWC	https://paperswithcode.com/paper/efficient-summarization-with-read-again-and
Repo
Framework

A Step from Probabilistic Programming to Cognitive Architectures


Title	A Step from Probabilistic Programming to Cognitive Architectures
Authors	Alexey Potapov
Abstract	Probabilistic programming is considered as a framework, in which basic components of cognitive architectures can be represented in unified and elegant fashion. At the same time, necessity of adopting some component of cognitive architectures for extending capabilities of probabilistic programming languages is pointed out. In particular, implicit specification of generative models via declaration of concepts and links between them is proposed, and usefulness of declarative knowledge for achieving efficient inference is briefly discussed.
Tasks	Probabilistic Programming
Published	2016-05-04
URL	http://arxiv.org/abs/1605.01180v1
PDF	http://arxiv.org/pdf/1605.01180v1.pdf
PWC	https://paperswithcode.com/paper/a-step-from-probabilistic-programming-to
Repo
Framework

Complexity of Discrete Energy Minimization Problems


Title	Complexity of Discrete Energy Minimization Problems
Authors	Mengtian Li, Alexander Shekhovtsov, Daniel Huber
Abstract	Discrete energy minimization is widely-used in computer vision and machine learning for problems such as MAP inference in graphical models. The problem, in general, is notoriously intractable, and finding the global optimal solution is known to be NP-hard. However, is it possible to approximate this problem with a reasonable ratio bound on the solution quality in polynomial time? We show in this paper that the answer is no. Specifically, we show that general energy minimization, even in the 2-label pairwise case, and planar energy minimization with three or more labels are exp-APX-complete. This finding rules out the existence of any approximation algorithm with a sub-exponential approximation ratio in the input size for these two problems, including constant factor approximations. Moreover, we collect and review the computational complexity of several subclass problems and arrange them on a complexity scale consisting of three major complexity classes – PO, APX, and exp-APX, corresponding to problems that are solvable, approximable, and inapproximable in polynomial time. Problems in the first two complexity classes can serve as alternative tractable formulations to the inapproximable ones. This paper can help vision researchers to select an appropriate model for an application or guide them in designing new algorithms.
Tasks
Published	2016-07-29
URL	http://arxiv.org/abs/1607.08905v1
PDF	http://arxiv.org/pdf/1607.08905v1.pdf
PWC	https://paperswithcode.com/paper/complexity-of-discrete-energy-minimization
Repo
Framework

Learning Power Spectrum Maps from Quantized Power Measurements


Title	Learning Power Spectrum Maps from Quantized Power Measurements
Authors	Daniel Romero, Seung-Jun Kim, Georgios B. Giannakis, Roberto Lopez-Valcarce
Abstract	Power spectral density (PSD) maps providing the distribution of RF power across space and frequency are constructed using power measurements collected by a network of low-cost sensors. By introducing linear compression and quantization to a small number of bits, sensor measurements can be communicated to the fusion center with minimal bandwidth requirements. Strengths of data- and model-driven approaches are combined to develop estimators capable of incorporating multiple forms of spectral and propagation prior information while fitting the rapid variations of shadow fading across space. To this end, novel nonparametric and semiparametric formulations are investigated. It is shown that PSD maps can be obtained using support vector machine-type solvers. In addition to batch approaches, an online algorithm attuned to real-time operation is developed. Numerical tests assess the performance of the novel algorithms.
Tasks	Quantization
Published	2016-06-07
URL	http://arxiv.org/abs/1606.02679v2
PDF	http://arxiv.org/pdf/1606.02679v2.pdf
PWC	https://paperswithcode.com/paper/learning-power-spectrum-maps-from-quantized
Repo
Framework

Rademacher Complexity Bounds for a Penalized Multiclass Semi-Supervised Algorithm


Title	Rademacher Complexity Bounds for a Penalized Multiclass Semi-Supervised Algorithm
Authors	Yury Maximov, Massih-Reza Amini, Zaid Harchaoui
Abstract	We propose Rademacher complexity bounds for multiclass classifiers trained with a two-step semi-supervised model. In the first step, the algorithm partitions the partially labeled data and then identifies dense clusters containing $\kappa$ predominant classes using the labeled training examples such that the proportion of their non-predominant classes is below a fixed threshold. In the second step, a classifier is trained by minimizing a margin empirical loss over the labeled training set and a penalization term measuring the disability of the learner to predict the $\kappa$ predominant classes of the identified clusters. The resulting data-dependent generalization error bound involves the margin distribution of the classifier, the stability of the clustering technique used in the first step and Rademacher complexity terms corresponding to partially labeled training data. Our theoretical result exhibit convergence rates extending those proposed in the literature for the binary case, and experimental results on different multiclass classification problems show empirical evidence that supports the theory.
Tasks
Published	2016-07-02
URL	http://arxiv.org/abs/1607.00567v3
PDF	http://arxiv.org/pdf/1607.00567v3.pdf
PWC	https://paperswithcode.com/paper/rademacher-complexity-bounds-for-a-penalized
Repo
Framework

A movie genre prediction based on Multivariate Bernoulli model and genre correlations


Title	A movie genre prediction based on Multivariate Bernoulli model and genre correlations
Authors	Eric Makita, Artem Lenskiy
Abstract	Movie ratings play an important role both in determining the likelihood of a potential viewer to watch the movie and in reflecting the current viewer satisfaction with the movie. They are available in several sources like the television guide, best-selling reference books, newspaper columns, and television programs. Furthermore, movie ratings are crucial for recommendation engines that track the behavior of all users and utilize the information to suggest items they might like. Movie ratings in most cases, thus, provide information that might be more important than movie feature-based data. It is intuitively appealing that information about the viewing preferences in movie genres is sufficient for predicting a genre of an unlabeled movie. In order to predict movie genres, we treat ratings as a feature vector, apply the Bernoulli event model to estimate the likelihood of a movies given genre, and evaluate the posterior probability of the genre of a given movie using the Bayes rule. The goal of the proposed technique is to efficiently use the movie ratings for the task of predicting movie genres. In our approach we attempted to answer the question: “Given the set of users who watched a movie, is it possible to predict the genre of a movie based on its ratings?” Our simulation results with MovieLens 100k data demonstrated the efficiency and accuracy of our proposed technique, achieving 59% prediction rate for exact prediction and 69% when including correlated genres.
Tasks
Published	2016-03-25
URL	http://arxiv.org/abs/1604.08608v1
PDF	http://arxiv.org/pdf/1604.08608v1.pdf
PWC	https://paperswithcode.com/paper/a-movie-genre-prediction-based-on
Repo
Framework

A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems


Title	A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems
Authors	Layla El Asri, Jing He, Kaheer Suleman
Abstract	User simulation is essential for generating enough data to train a statistical spoken dialogue system. Previous models for user simulation suffer from several drawbacks, such as the inability to take dialogue history into account, the need of rigid structure to ensure coherent user behaviour, heavy dependence on a specific domain, the inability to output several user intentions during one dialogue turn, or the requirement of a summarized action space for tractability. This paper introduces a data-driven user simulator based on an encoder-decoder recurrent neural network. The model takes as input a sequence of dialogue contexts and outputs a sequence of dialogue acts corresponding to user intentions. The dialogue contexts include information about the machine acts and the status of the user goal. We show on the Dialogue State Tracking Challenge 2 (DSTC2) dataset that the sequence-to-sequence model outperforms an agenda-based simulator and an n-gram simulator, according to F-score. Furthermore, we show how this model can be used on the original action space and thereby models user behaviour with finer granularity.
Tasks	Dialogue State Tracking, Spoken Dialogue Systems
Published	2016-06-30
URL	http://arxiv.org/abs/1607.00070v1
PDF	http://arxiv.org/pdf/1607.00070v1.pdf
PWC	https://paperswithcode.com/paper/a-sequence-to-sequence-model-for-user
Repo
Framework

Hash2Vec, Feature Hashing for Word Embeddings


Title	Hash2Vec, Feature Hashing for Word Embeddings
Authors	Luis Argerich, Joaquín Torré Zaffaroni, Matías J Cano
Abstract	In this paper we propose the application of feature hashing to create word embeddings for natural language processing. Feature hashing has been used successfully to create document vectors in related tasks like document classification. In this work we show that feature hashing can be applied to obtain word embeddings in linear time with the size of the data. The results show that this algorithm, that does not need training, is able to capture the semantic meaning of words. We compare the results against GloVe showing that they are similar. As far as we know this is the first application of feature hashing to the word embeddings problem and the results indicate this is a scalable technique with practical results for NLP applications.
Tasks	Document Classification, Word Embeddings
Published	2016-08-31
URL	http://arxiv.org/abs/1608.08940v1
PDF	http://arxiv.org/pdf/1608.08940v1.pdf
PWC	https://paperswithcode.com/paper/hash2vec-feature-hashing-for-word-embeddings
Repo
Framework

Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfer


Title	Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfer
Authors	Coline Devin, Abhishek Gupta, Trevor Darrell, Pieter Abbeel, Sergey Levine
Abstract	Reinforcement learning (RL) can automate a wide variety of robotic skills, but learning each new skill requires considerable real-world data collection and manual representation engineering to design policy classes or features. Using deep reinforcement learning to train general purpose neural network policies alleviates some of the burden of manual representation engineering by using expressive policy classes, but exacerbates the challenge of data collection, since such methods tend to be less efficient than RL with low-dimensional, hand-designed representations. Transfer learning can mitigate this problem by enabling us to transfer information from one skill to another and even from one robot to another. We show that neural network policies can be decomposed into “task-specific” and “robot-specific” modules, where the task-specific modules are shared across robots, and the robot-specific modules are shared across all tasks on that robot. This allows for sharing task information, such as perception, between robots and sharing robot information, such as dynamics and kinematics, between tasks. We exploit this decomposition to train mix-and-match modules that can solve new robot-task combinations that were not seen during training. Using a novel neural network architecture, we demonstrate the effectiveness of our transfer method for enabling zero-shot generalization with a variety of robots and tasks in simulation for both visual and non-visual tasks.
Tasks	Transfer Learning
Published	2016-09-22
URL	http://arxiv.org/abs/1609.07088v1
PDF	http://arxiv.org/pdf/1609.07088v1.pdf
PWC	https://paperswithcode.com/paper/learning-modular-neural-network-policies-for
Repo
Framework

Single-View and Multi-View Depth Fusion


Title	Single-View and Multi-View Depth Fusion
Authors	José M. Fácil, Alejo Concha, Luis Montesano, Javier Civera
Abstract	Dense and accurate 3D mapping from a monocular sequence is a key technology for several applications and still an open research area. This paper leverages recent results on single-view CNN-based depth estimation and fuses them with multi-view depth estimation. Both approaches present complementary strengths. Multi-view depth is highly accurate but only in high-texture areas and high-parallax cases. Single-view depth captures the local structure of mid-level regions, including texture-less areas, but the estimated depth lacks global coherence. The single and multi-view fusion we propose is challenging in several aspects. First, both depths are related by a deformation that depends on the image content. Second, the selection of multi-view points of high accuracy might be difficult for low-parallax configurations. We present contributions for both problems. Our results in the public datasets of NYUv2 and TUM shows that our algorithm outperforms the individual single and multi-view approaches. A video showing the key aspects of mapping in our Single and Multi-view depth proposal is available at https://youtu.be/ipc5HukTb4k
Tasks	Depth Estimation
Published	2016-11-22
URL	http://arxiv.org/abs/1611.07245v2
PDF	http://arxiv.org/pdf/1611.07245v2.pdf
PWC	https://paperswithcode.com/paper/single-view-and-multi-view-depth-fusion
Repo
Framework

ActionFlowNet: Learning Motion Representation for Action Recognition


Title	ActionFlowNet: Learning Motion Representation for Action Recognition
Authors	Joe Yue-Hei Ng, Jonghyun Choi, Jan Neumann, Larry S. Davis
Abstract	Even with the recent advances in convolutional neural networks (CNN) in various visual recognition tasks, the state-of-the-art action recognition system still relies on hand crafted motion feature such as optical flow to achieve the best performance. We propose a multitask learning model ActionFlowNet to train a single stream network directly from raw pixels to jointly estimate optical flow while recognizing actions with convolutional neural networks, capturing both appearance and motion in a single model. We additionally provide insights to how the quality of the learned optical flow affects the action recognition. Our model significantly improves action recognition accuracy by a large margin 31% compared to state-of-the-art CNN-based action recognition models trained without external large scale data and additional optical flow input. Without pretraining on large external labeled datasets, our model, by well exploiting the motion information, achieves competitive recognition accuracy to the models trained with large labeled datasets such as ImageNet and Sport-1M.
Tasks	Optical Flow Estimation, Temporal Action Localization
Published	2016-12-09
URL	http://arxiv.org/abs/1612.03052v3
PDF	http://arxiv.org/pdf/1612.03052v3.pdf
PWC	https://paperswithcode.com/paper/actionflownet-learning-motion-representation
Repo
Framework

LSTM-based Mixture-of-Experts for Knowledge-Aware Dialogues


Title	LSTM-based Mixture-of-Experts for Knowledge-Aware Dialogues
Authors	Phong Le, Marc Dymetman, Jean-Michel Renders
Abstract	We introduce an LSTM-based method for dynamically integrating several word-prediction experts to obtain a conditional language model which can be good simultaneously at several subtasks. We illustrate this general approach with an application to dialogue where we integrate a neural chat model, good at conversational aspects, with a neural question-answering model, good at retrieving precise information from a knowledge-base, and show how the integration combines the strengths of the independent components. We hope that this focused contribution will attract attention on the benefits of using such mixtures of experts in NLP.
Tasks	Language Modelling, Question Answering
Published	2016-05-05
URL	http://arxiv.org/abs/1605.01652v1
PDF	http://arxiv.org/pdf/1605.01652v1.pdf
PWC	https://paperswithcode.com/paper/lstm-based-mixture-of-experts-for-knowledge
Repo
Framework