January 27, 2020

3059 words 15 mins read

Paper Group ANR 1134

The University of Edinburgh’s Submissions to the WMT19 News Translation Task. A review on Neural Turing Machine. Finite-Sum Smooth Optimization with SARAH. A Learning Framework for Distribution-Based Game-Theoretic Solution Concepts. Common Mode Patterns for Supervised Tensor Subspace Learning. Robust Environmental Sound Recognition with Sparse Key …

The University of Edinburgh’s Submissions to the WMT19 News Translation Task


Title	The University of Edinburgh’s Submissions to the WMT19 News Translation Task
Authors	Rachel Bawden, Nikolay Bogoychev, Ulrich Germann, Roman Grundkiewicz, Faheem Kirefu, Antonio Valerio Miceli Barone, Alexandra Birch
Abstract	The University of Edinburgh participated in the WMT19 Shared Task on News Translation in six language directions: English-to-Gujarati, Gujarati-to-English, English-to-Chinese, Chinese-to-English, German-to-English, and English-to-Czech. For all translation directions, we created or used back-translations of monolingual data in the target language as additional synthetic training data. For English-Gujarati, we also explored semi-supervised MT with cross-lingual language model pre-training, and translation pivoting through Hindi. For translation to and from Chinese, we investigated character-based tokenisation vs. sub-word segmentation of Chinese text. For German-to-English, we studied the impact of vast amounts of back-translated training data on translation quality, gaining a few additional insights over Edunov et al. (2018). For English-to-Czech, we compared different pre-processing and tokenisation regimes.
Tasks	Language Modelling
Published	2019-07-12
URL	https://arxiv.org/abs/1907.05854v1
PDF	https://arxiv.org/pdf/1907.05854v1.pdf
PWC	https://paperswithcode.com/paper/the-university-of-edinburghs-submissions-to
Repo
Framework

A review on Neural Turing Machine


Title	A review on Neural Turing Machine
Authors	Soroor Malekmohammadi Faradonbeh, Faramarz Safi-Esfahani
Abstract	One of the major objectives of Artificial Intelligence is to design learning algorithms that are executed on a general purposes computational machines such as human brain. Neural Turing Machine (NTM) is a step towards realizing such a computational machine. The attempt is made here to run a systematic review on Neural Turing Machine. First, the mind-map and taxonomy of machine learning, neural networks, and Turing machine are introduced. Next, NTM is inspected in terms of concepts, structure, variety of versions, implemented tasks, comparisons, etc. Finally, the paper discusses on issues and ends up with several future works.
Tasks
Published	2019-04-10
URL	http://arxiv.org/abs/1904.05061v1
PDF	http://arxiv.org/pdf/1904.05061v1.pdf
PWC	https://paperswithcode.com/paper/a-review-on-neural-turing-machine
Repo
Framework

Finite-Sum Smooth Optimization with SARAH


Title	Finite-Sum Smooth Optimization with SARAH
Authors	Lam M. Nguyen, Marten van Dijk, Dzung T. Phan, Phuong Ha Nguyen, Tsui-Wei Weng, Jayant R. Kalagnanam
Abstract	The total complexity (measured as the total number of gradient computations) of a stochastic first-order optimization algorithm that finds a first-order stationary point of a finite-sum smooth nonconvex objective function $F(w)=\frac{1}{n} \sum_{i=1}^n f_i(w)$ has been proven to be at least $\Omega(\sqrt{n}/\epsilon)$ for $n \leq \mathcal{O}(\epsilon^{-2})$ where $\epsilon$ denotes the attained accuracy $\mathbb{E}[ \nabla F(\tilde{w})^2] \leq \epsilon$ for the outputted approximation $\tilde{w}$ (Fang et al., 2018). In this paper, we provide a convergence analysis for a slightly modified version of the SARAH algorithm (Nguyen et al., 2017a;b) and achieve total complexity that matches the lower-bound worst case complexity in (Fang et al., 2018) up to a constant factor when $n \leq \mathcal{O}(\epsilon^{-2})$ for nonconvex problems. For convex optimization, we propose SARAH++ with sublinear convergence for general convex and linear convergence for strongly convex problems; and we provide a practical version for which numerical experiments on various datasets show an improved performance.
Tasks
Published	2019-01-22
URL	http://arxiv.org/abs/1901.07648v2
PDF	http://arxiv.org/pdf/1901.07648v2.pdf
PWC	https://paperswithcode.com/paper/optimal-finite-sum-smooth-non-convex
Repo
Framework

A Learning Framework for Distribution-Based Game-Theoretic Solution Concepts


Title	A Learning Framework for Distribution-Based Game-Theoretic Solution Concepts
Authors	Tushant Jha, Yair Zick
Abstract	The past few years have seen several works on learning economic solutions from data; these include optimal auction design, function optimization, stable payoffs in cooperative games and more. In this work, we provide a unified learning-theoretic methodology for modeling such problems, and establish tools for determining whether a given economic solution concept can be learned from data. Our learning theoretic framework generalizes a notion of function space dimension – the graph dimension – adapting it to the solution concept learning domain. We identify sufficient conditions for the PAC learnability of solution concepts, and show that results in existing works can be immediately derived using our methodology. Finally, we apply our methods in other economic domains, yielding a novel notion of PAC competitive equilibrium and PAC Condorcet winners.
Tasks
Published	2019-03-20
URL	https://arxiv.org/abs/1903.08322v2
PDF	https://arxiv.org/pdf/1903.08322v2.pdf
PWC	https://paperswithcode.com/paper/a-learning-framework-for-distribution-based
Repo
Framework

Common Mode Patterns for Supervised Tensor Subspace Learning


Title	Common Mode Patterns for Supervised Tensor Subspace Learning
Authors	Konstantinos Makantasis, Anastasios Doulamis, Nikolaos Doulamis, Athanasios Voulodimos
Abstract	In this work we propose a method for reducing the dimensionality of tensor objects in a binary classification framework. The proposed Common Mode Patterns method takes into consideration the labels’ information, and ensures that tensor objects that belong to different classes do not share common features after the reduction of their dimensionality. We experimentally validate the proposed supervised subspace learning technique and compared it against Multilinear Principal Component Analysis using a publicly available hyperspectral imaging dataset. Experimental results indicate that the proposed CMP method can efficiently reduce the dimensionality of tensor objects, while, at the same time, increasing the inter-class separability.
Tasks
Published	2019-02-06
URL	http://arxiv.org/abs/1902.02075v1
PDF	http://arxiv.org/pdf/1902.02075v1.pdf
PWC	https://paperswithcode.com/paper/common-mode-patterns-for-supervised-tensor
Repo
Framework

Robust Environmental Sound Recognition with Sparse Key-point Encoding and Efficient Multi-spike Learning


Title	Robust Environmental Sound Recognition with Sparse Key-point Encoding and Efficient Multi-spike Learning
Authors	Qiang Yu, Yanli Yao, Longbiao Wang, Huajin Tang, Jianwu Dang, Kay Chen Tan
Abstract	The capability for environmental sound recognition (ESR) can determine the fitness of individuals in a way to avoid dangers or pursue opportunities when critical sound events occur. It still remains mysterious about the fundamental principles of biological systems that result in such a remarkable ability. Additionally, the practical importance of ESR has attracted an increasing amount of research attention, but the chaotic and non-stationary difficulties continue to make it a challenging task. In this study, we propose a spike-based framework from a more brain-like perspective for the ESR task. Our framework is a unifying system with a consistent integration of three major functional parts which are sparse encoding, efficient learning and robust readout. We first introduce a simple sparse encoding where key-points are used for feature representation, and demonstrate its generalization to both spike and non-spike based systems. Then, we evaluate the learning properties of different learning rules in details with our contributions being added for improvements. Our results highlight the advantages of the multi-spike learning, providing a selection reference for various spike-based developments. Finally, we combine the multi-spike readout with the other parts to form a system for ESR. Experimental results show that our framework performs the best as compared to other baseline approaches. In addition, we show that our spike-based framework has several advantageous characteristics including early decision making, small dataset acquiring and ongoing dynamic processing. Our framework is the first attempt to apply the multi-spike characteristic of nervous neurons to ESR. The outstanding performance of our approach would potentially contribute to draw more research efforts to push the boundaries of spike-based paradigm to a new horizon.
Tasks	Decision Making
Published	2019-02-04
URL	http://arxiv.org/abs/1902.01094v1
PDF	http://arxiv.org/pdf/1902.01094v1.pdf
PWC	https://paperswithcode.com/paper/robust-environmental-sound-recognition-with
Repo
Framework

Large Scale Open-Set Deep Logo Detection


Title	Large Scale Open-Set Deep Logo Detection
Authors	Muhammet Bastan, Hao-Yu Wu, Tian Cao, Bhargava Kota, Mehmet Tek
Abstract	We present an open-set logo detection (OSLD) system, which can detect (localize and recognize) any number of unseen logo classes without re-training; it only requires a small set of canonical logo images for each logo class. We achieve this using a two-stage approach: (1) Generic logo detection to detect candidate logo regions in an image. (2) Logo matching for matching the detected logo regions to a set of canonical logo images to recognize them. We also introduce a ‘simple deep metric learning’ (SDML) framework that outperformed more complicated ensemble and attention models and boosted the logo matching accuracy. Furthermore, we constructed a new open-set logo detection dataset with thousands of logo classes, and will release it for research purposes. We demonstrate the effectiveness of OSLD on our dataset and on the standard Flickr-32 logo dataset, outperforming the state-of-the-art open-set and closed-set logo detection methods by a large margin.
Tasks	Metric Learning
Published	2019-11-18
URL	https://arxiv.org/abs/1911.07440v1
PDF	https://arxiv.org/pdf/1911.07440v1.pdf
PWC	https://paperswithcode.com/paper/large-scale-open-set-deep-logo-detection
Repo
Framework

Equal Opportunity in Online Classification with Partial Feedback


Title	Equal Opportunity in Online Classification with Partial Feedback
Authors	Yahav Bechavod, Katrina Ligett, Aaron Roth, Bo Waggoner, Zhiwei Steven Wu
Abstract	We study an online classification problem with partial feedback in which individuals arrive one at a time from a fixed but unknown distribution, and must be classified as positive or negative. Our algorithm only observes the true label of an individual if they are given a positive classification. This setting captures many classification problems for which fairness is a concern: for example, in criminal recidivism prediction, recidivism is only observed if the inmate is released; in lending applications, loan repayment is only observed if the loan is granted. We require that our algorithms satisfy common statistical fairness constraints (such as equalizing false positive or negative rates — introduced as “equal opportunity” in Hardt et al. (2016)) at every round, with respect to the underlying distribution. We give upper and lower bounds characterizing the cost of this constraint in terms of the regret rate (and show that it is mild), and give an oracle efficient algorithm that achieves the upper bound.
Tasks
Published	2019-02-06
URL	http://arxiv.org/abs/1902.02242v1
PDF	http://arxiv.org/pdf/1902.02242v1.pdf
PWC	https://paperswithcode.com/paper/equal-opportunity-in-online-classification
Repo
Framework

Learning Robust Representations with Graph Denoising Policy Network


Title	Learning Robust Representations with Graph Denoising Policy Network
Authors	Lu Wang, Wenchao Yu, Wei Wang, Wei Cheng, Wei Zhang, Hongyuan Zha, Xiaofeng He, Haifeng Chen
Abstract	Graph representation learning, aiming to learn low-dimensional representations which capture the geometric dependencies between nodes in the original graph, has gained increasing popularity in a variety of graph analysis tasks, including node classification and link prediction. Existing representation learning methods based on graph neural networks and their variants rely on the aggregation of neighborhood information, which makes it sensitive to noises in the graph. In this paper, we propose Graph Denoising Policy Network (short for GDPNet) to learn robust representations from noisy graph data through reinforcement learning. GDPNet first selects signal neighborhoods for each node, and then aggregates the information from the selected neighborhoods to learn node representations for the down-stream tasks. Specifically, in the signal neighborhood selection phase, GDPNet optimizes the neighborhood for each target node by formulating the process of removing noisy neighborhoods as a Markov decision process and learning a policy with task-specific rewards received from the representation learning phase. In the representation learning phase, GDPNet aggregates features from signal neighbors to generate node representations for down-stream tasks, and provides task-specific rewards to the signal neighbor selection phase. These two phases are jointly trained to select optimal sets of neighbors for target nodes with maximum cumulative task-specific rewards, and to learn robust representations for nodes. Experimental results on node classification task demonstrate the effectiveness of GDNet, outperforming the state-of-the-art graph representation learning methods on several well-studied datasets. Additionally, GDPNet is mathematically equivalent to solving the submodular maximizing problem, which theoretically guarantees the best approximation to the optimal solution with GDPNet.
Tasks	Denoising, Graph Representation Learning, Link Prediction, Node Classification, Representation Learning
Published	2019-10-04
URL	https://arxiv.org/abs/1910.01784v1
PDF	https://arxiv.org/pdf/1910.01784v1.pdf
PWC	https://paperswithcode.com/paper/learning-robust-representations-with-graph
Repo
Framework

Teaching with IMPACT


Title	Teaching with IMPACT
Authors	Carl Trimbach, Michael Littman
Abstract	Like many problems in AI in their general form, supervised learning is computationally intractable. We hypothesize that an important reason humans can learn highly complex and varied concepts, in spite of the computational difficulty, is that they benefit tremendously from experienced and insightful teachers. This paper proposes a new learning framework that provides a role for a knowledgeable, benevolent teacher to guide the process of learning a target concept in a series of “curricular” phases or rounds. In each round, the teacher’s role is to act as a moderator, exposing the learner to a subset of the available training data to move it closer to mastering the target concept. Via both theoretical and empirical evidence, we argue that this framework enables simple, efficient learners to acquire very complex concepts from examples. In particular, we provide multiple examples of concept classes that are known to be unlearnable in the standard PAC setting along with provably efficient algorithms for learning them in our extended setting. A key focus of our work is the ability to learn complex concepts on top of simpler, previously learned, concepts—a direction with the potential of creating more competent artificial agents.
Tasks
Published	2019-03-14
URL	http://arxiv.org/abs/1903.06209v1
PDF	http://arxiv.org/pdf/1903.06209v1.pdf
PWC	https://paperswithcode.com/paper/teaching-with-impact
Repo
Framework

Capturing Greater Context for Question Generation


Title	Capturing Greater Context for Question Generation
Authors	Luu Anh Tuan, Darsh J Shah, Regina Barzilay
Abstract	Automatic question generation can benefit many applications ranging from dialogue systems to reading comprehension. While questions are often asked with respect to long documents, there are many challenges with modeling such long documents. Many existing techniques generate questions by effectively looking at one sentence at a time, leading to questions that are easy and not reflective of the human process of question generation. Our goal is to incorporate interactions across multiple sentences to generate realistic questions for long documents. In order to link a broad document context to the target answer, we represent the relevant context via a multi-stage attention mechanism, which forms the foundation of a sequence to sequence model. We outperform state-of-the-art methods on question generation on three question-answering datasets – SQuAD, MS MARCO and NewsQA.
Tasks	Question Answering, Question Generation, Reading Comprehension
Published	2019-10-22
URL	https://arxiv.org/abs/1910.10274v1
PDF	https://arxiv.org/pdf/1910.10274v1.pdf
PWC	https://paperswithcode.com/paper/capturing-greater-context-for-question
Repo
Framework

Competing Bandits in Matching Markets


Title	Competing Bandits in Matching Markets
Authors	Lydia T. Liu, Horia Mania, Michael I. Jordan
Abstract	Stable matching, a classical model for two-sided markets, has long been studied with little consideration for how each side’s preferences are learned. With the advent of massive online markets powered by data-driven matching platforms, it has become necessary to better understand the interplay between learning and market objectives. We propose a statistical learning model in which one side of the market does not have a priori knowledge about its preferences for the other side and is required to learn these from stochastic rewards. Our model extends the standard multi-armed bandits framework to multiple players, with the added feature that arms have preferences over players. We study both centralized and decentralized approaches to this problem and show surprising exploration-exploitation trade-offs compared to the single player multi-armed bandits setting.
Tasks	Multi-Armed Bandits
Published	2019-06-12
URL	https://arxiv.org/abs/1906.05363v1
PDF	https://arxiv.org/pdf/1906.05363v1.pdf
PWC	https://paperswithcode.com/paper/competing-bandits-in-matching-markets
Repo
Framework

Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition


Title	Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition
Authors	Ran He, Jie Cao, Lingxiao Song, Zhenan Sun, Tieniu Tan
Abstract	Near infrared-visible (NIR-VIS) heterogeneous face recognition refers to the process of matching NIR to VIS face images. Current heterogeneous methods try to extend VIS face recognition methods to the NIR spectrum by synthesizing VIS images from NIR images. However, due to self-occlusion and sensing gap, NIR face images lose some visible lighting contents so that they are always incomplete compared to VIS face images. This paper models high resolution heterogeneous face synthesis as a complementary combination of two components, a texture inpainting component and pose correction component. The inpainting component synthesizes and inpaints VIS image textures from NIR image textures. The correction component maps any pose in NIR images to a frontal pose in VIS images, resulting in paired NIR and VIS textures. A warping procedure is developed to integrate the two components into an end-to-end deep network. A fine-grained discriminator and a wavelet-based discriminator are designed to supervise intra-class variance and visual quality respectively. One UV loss, two adversarial losses and one pixel loss are imposed to ensure synthesis results. We demonstrate that by attaching the correction component, we can simplify heterogeneous face synthesis from one-to-many unpaired image translation to one-to-one paired image translation, and minimize spectral and pose discrepancy during heterogeneous recognition. Extensive experimental results show that our network not only generates high-resolution VIS face images and but also facilitates the accuracy improvement of heterogeneous face recognition.
Tasks	Face Generation, Face Recognition, Facial Inpainting, Heterogeneous Face Recognition
Published	2019-02-10
URL	http://arxiv.org/abs/1902.03565v1
PDF	http://arxiv.org/pdf/1902.03565v1.pdf
PWC	https://paperswithcode.com/paper/cross-spectral-face-completion-for-nir-vis
Repo
Framework

Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation


Title	Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulation
Authors	Benjamin Beyret, Ali Shafti, A. Aldo Faisal
Abstract	Robotic systems are ever more capable of automation and fulfilment of complex tasks, particularly with reliance on recent advances in intelligent systems, deep learning and artificial intelligence. However, as robots and humans come closer in their interactions, the matter of interpretability, or explainability of robot decision-making processes for the human grows in importance. A successful interaction and collaboration will only take place through mutual understanding of underlying representations of the environment and the task at hand. This is currently a challenge in deep learning systems. We present a hierarchical deep reinforcement learning system, consisting of a low-level agent handling the large actions/states space of a robotic system efficiently, by following the directives of a high-level agent which is learning the high-level dynamics of the environment and task. This high-level agent forms a representation of the world and task at hand that is interpretable for a human operator. The method, which we call Dot-to-Dot, is tested on a MuJoCo-based model of the Fetch Robotics Manipulator, as well as a Shadow Hand, to test its performance. Results show efficient learning of complex actions/states spaces by the low-level agent, and an interpretable representation of the task and decision-making process learned by the high-level agent.
Tasks	Decision Making, Hierarchical Reinforcement Learning
Published	2019-04-14
URL	https://arxiv.org/abs/1904.06703v2
PDF	https://arxiv.org/pdf/1904.06703v2.pdf
PWC	https://paperswithcode.com/paper/dot-to-dot-achieving-structured-robotic
Repo
Framework

A General Framework for Saliency Detection Methods


Title	A General Framework for Saliency Detection Methods
Authors	Fateme Mostafaie, Zahra Nabizadeh, Nader Karimi, Shadrokh Samavi
Abstract	Saliency detection is one of the most challenging problems in the fields of image analysis and computer vision. Many approaches propose different architectures based on the psychological and biological properties of the human visual attention system. However, there is not still an abstract framework, which summarized the existed methods. In this paper, we offered a general framework for saliency models, which consists of five main steps: pre-processing, feature extraction, saliency map generation, saliency map combination, and post-processing. Also, we study different saliency models containing each level and compare their performance together. This framework helps researchers to have a comprehensive view of studying new methods.
Tasks	Saliency Detection
Published	2019-12-27
URL	https://arxiv.org/abs/1912.12027v1
PDF	https://arxiv.org/pdf/1912.12027v1.pdf
PWC	https://paperswithcode.com/paper/a-general-framework-for-saliency-detection
Repo
Framework