January 28, 2020

2954 words 14 mins read

Paper Group ANR 806

CUNI Systems for the Unsupervised News Translation Task in WMT 2019. MCTS-based Automated Negotiation Agent. Ensemble Clustering for Graphs: Comparisons and Applications. Fair Near Neighbor Search: Independent Range Sampling in High Dimensions. Dealing with Sparse Rewards in Reinforcement Learning. Transferable End-to-End Aspect-based Sentiment Ana …

CUNI Systems for the Unsupervised News Translation Task in WMT 2019


Title	CUNI Systems for the Unsupervised News Translation Task in WMT 2019
Authors	Ivana Kvapilíková, Dominik Macháček, Ondřej Bojar
Abstract	In this paper we describe the CUNI translation system used for the unsupervised news shared task of the ACL 2019 Fourth Conference on Machine Translation (WMT19). We follow the strategy of Artexte et al. (2018b), creating a seed phrase-based system where the phrase table is initialized from cross-lingual embedding mappings trained on monolingual data, followed by a neural machine translation system trained on synthetic parallel data. The synthetic corpus was produced from a monolingual corpus by a tuned PBMT model refined through iterative back-translation. We further focus on the handling of named entities, i.e. the part of vocabulary where the cross-lingual embedding mapping suffers most. Our system reaches a BLEU score of 15.3 on the German-Czech WMT19 shared task.
Tasks	Machine Translation
Published	2019-07-29
URL	https://arxiv.org/abs/1907.12664v1
PDF	https://arxiv.org/pdf/1907.12664v1.pdf
PWC	https://paperswithcode.com/paper/cuni-systems-for-the-unsupervised-news
Repo
Framework

MCTS-based Automated Negotiation Agent


Title	MCTS-based Automated Negotiation Agent
Authors	Cédric Buron, Zahia Guessoum, Sylvain Ductor
Abstract	This paper introduces a new negotiating agent model for automated negotiation. We focus on applications without time pressure with multidi-mensional negotiation on both continuous and discrete domains. The agent bidding strategy relies on Monte Carlo Tree Search, which is a trendy method since it has been used with success on games with high branching factor such as Go. It also exploits opponent modeling techniques thanks to Gaussian process regression and Bayesian learning. Evaluation is done by confronting the existing agents that are able to negotiate in such context: Random Walker, Tit-for-tat and Nice Tit-for-Tat. None of those agents succeeds in beating our agent. Also, the modular and adaptive nature of our approach is a huge advantage when it comes to optimize it in specific applicative contexts.
Tasks
Published	2019-09-12
URL	https://arxiv.org/abs/1909.09461v1
PDF	https://arxiv.org/pdf/1909.09461v1.pdf
PWC	https://paperswithcode.com/paper/mcts-based-automated-negotiation-agent-1
Repo
Framework

Ensemble Clustering for Graphs: Comparisons and Applications


Title	Ensemble Clustering for Graphs: Comparisons and Applications
Authors	Valérie Poulin, François Théberge
Abstract	We recently proposed a new ensemble clustering algorithm for graphs (ECG) based on the concept of consensus clustering. We validated our approach by replicating a study comparing graph clustering algorithms over benchmark graphs, showing that ECG outperforms the leading algorithms. In this paper, we extend our comparison by considering a wider range of parameters for the benchmark, generating graphs with different properties. We provide new experimental results showing that the ECG algorithm alleviates the well-known resolution limit issue, and that it leads to better stability of the partitions. We also illustrate how the ensemble obtained with ECG can be used to quantify the presence of community structure in the graph, and to zoom in on the sub-graph most closely associated with seed vertices. Finally, we illustrate further applications of ECG by comparing it to previous results for community detection on weighted graphs, and community-aware anomaly detection.
Tasks	Anomaly Detection, Community Detection, Graph Clustering
Published	2019-03-19
URL	http://arxiv.org/abs/1903.08012v1
PDF	http://arxiv.org/pdf/1903.08012v1.pdf
PWC	https://paperswithcode.com/paper/ensemble-clustering-for-graphs-comparisons
Repo
Framework

Fair Near Neighbor Search: Independent Range Sampling in High Dimensions


Title	Fair Near Neighbor Search: Independent Range Sampling in High Dimensions
Authors	Martin Aumüller, Rasmus Pagh, Francesco Silvestri
Abstract	Similarity search is a fundamental algorithmic primitive, widely used in many computer science disciplines. There are several variants of the similarity search problem, and one of the most relevant is the $r$-near neighbor ($r$-NN) problem: given a radius $r>0$ and a set of points $S$, construct a data structure that, for any given query point $q$, returns a point $p$ within distance at most $r$ from $q$. In this paper, we study the $r$-NN problem in the light of fairness. We consider fairness in the sense of equal opportunity: all points that are within distance $r$ from the query should have the same probability to be returned. Locality sensitive hashing (LSH), the most common approach to similarity search in high dimensions, does not provide such a fairness guarantee. To address this, we propose efficient data structures for $r$-NN where all points in $S$ that are near $q$ have the same probability to be selected and returned by the query. Specifically, we first propose a black-box approach that, given any LSH scheme, constructs a data structure for uniformly sampling points in the neighborhood of a query. Then, we develop a data structure for fair similarity search under inner product, which requires nearly-linear space and exploits locality sensitive filters.
Tasks
Published	2019-06-05
URL	https://arxiv.org/abs/1906.01859v1
PDF	https://arxiv.org/pdf/1906.01859v1.pdf
PWC	https://paperswithcode.com/paper/fair-near-neighbor-search-independent-range
Repo
Framework

Dealing with Sparse Rewards in Reinforcement Learning


Title	Dealing with Sparse Rewards in Reinforcement Learning
Authors	Joshua Hare
Abstract	Successfully navigating a complex environment to obtain a desired outcome is a difficult task, that up to recently was believed to be capable only by humans. This perception has been broken down over time, especially with the introduction of deep reinforcement learning, which has greatly increased the difficulty of tasks that can be automated. However, for traditional reinforcement learning agents this requires an environment to be able to provide frequent extrinsic rewards, which are not known or accessible for many real-world environments. This project aims to explore and contrast existing reinforcement learning solutions that circumnavigate the difficulties of an environment that provide sparse rewards. Different reinforcement solutions will be implemented over a several video game environments with varying difficulty and varying frequency of rewards, as to properly investigate the applicability of these solutions. This project introduces a novel reinforcement learning solution by combining aspects of two existing state of the art sparse reward solutions, curiosity driven exploration and unsupervised auxiliary tasks.
Tasks
Published	2019-10-21
URL	https://arxiv.org/abs/1910.09281v2
PDF	https://arxiv.org/pdf/1910.09281v2.pdf
PWC	https://paperswithcode.com/paper/dealing-with-sparse-rewards-in-reinforcement
Repo
Framework

Transferable End-to-End Aspect-based Sentiment Analysis with Selective Adversarial Learning


Title	Transferable End-to-End Aspect-based Sentiment Analysis with Selective Adversarial Learning
Authors	Zheng Li, Xin Li, Ying Wei, Lidong Bing, Yu Zhang, Qiang Yang
Abstract	Joint extraction of aspects and sentiments can be effectively formulated as a sequence labeling problem. However, such formulation hinders the effectiveness of supervised methods due to the lack of annotated sequence data in many domains. To address this issue, we firstly explore an unsupervised domain adaptation setting for this task. Prior work can only use common syntactic relations between aspect and opinion words to bridge the domain gaps, which highly relies on external linguistic resources. To resolve it, we propose a novel Selective Adversarial Learning (SAL) method to align the inferred correlation vectors that automatically capture their latent relations. The SAL method can dynamically learn an alignment weight for each word such that more important words can possess higher alignment weights to achieve fine-grained (word-level) adaptation. Empirically, extensive experiments demonstrate the effectiveness of the proposed SAL method.
Tasks	Aspect-Based Sentiment Analysis, Domain Adaptation, Sentiment Analysis, Unsupervised Domain Adaptation
Published	2019-10-31
URL	https://arxiv.org/abs/1910.14192v1
PDF	https://arxiv.org/pdf/1910.14192v1.pdf
PWC	https://paperswithcode.com/paper/transferable-end-to-end-aspect-based
Repo
Framework

A Comparative Study on Vocabulary Reduction for Phrase Table Smoothing


Title	A Comparative Study on Vocabulary Reduction for Phrase Table Smoothing
Authors	Yunsu Kim, Andreas Guta, Joern Wuebker, Hermann Ney
Abstract	This work systematically analyzes the smoothing effect of vocabulary reduction for phrase translation models. We extensively compare various word-level vocabularies to show that the performance of smoothing is not significantly affected by the choice of vocabulary. This result provides empirical evidence that the standard phrase translation model is extremely sparse. Our experiments also reveal that vocabulary reduction is more effective for smoothing large-scale phrase tables.
Tasks
Published	2019-01-06
URL	http://arxiv.org/abs/1901.01574v1
PDF	http://arxiv.org/pdf/1901.01574v1.pdf
PWC	https://paperswithcode.com/paper/a-comparative-study-on-vocabulary-reduction
Repo
Framework

Look-up and Adapt: A One-shot Semantic Parser


Title	Look-up and Adapt: A One-shot Semantic Parser
Authors	Zhichu Lu, Forough Arabshahi, Igor Labutov, Tom Mitchell
Abstract	Computing devices have recently become capable of interacting with their end users via natural language. However, they can only operate within a limited “supported” domain of discourse and fail drastically when faced with an out-of-domain utterance, mainly due to the limitations of their semantic parser. In this paper, we propose a semantic parser that generalizes to out-of-domain examples by learning a general strategy for parsing an unseen utterance through adapting the logical forms of seen utterances, instead of learning to generate a logical form from scratch. Our parser maintains a memory consisting of a representative subset of the seen utterances paired with their logical forms. Given an unseen utterance, our parser works by looking up a similar utterance from the memory and adapting its logical form until it fits the unseen utterance. Moreover, we present a data generation strategy for constructing utterance-logical form pairs from different domains. Our results show an improvement of up to 68.8% on one-shot parsing under two different evaluation settings compared to the baselines.
Tasks
Published	2019-10-27
URL	https://arxiv.org/abs/1910.12197v1
PDF	https://arxiv.org/pdf/1910.12197v1.pdf
PWC	https://paperswithcode.com/paper/look-up-and-adapt-a-one-shot-semantic-parser
Repo
Framework

Variational Bayesian modelling of mixed-effects


Title	Variational Bayesian modelling of mixed-effects
Authors	Jean Daunizeau
Abstract	This note is concerned with an accurate and computationally efficient variational bayesian treatment of mixed-effects modelling. We focus on group studies, i.e. empirical studies that report multiple measurements acquired in multiple subjects. When approached from a bayesian perspective, such mixed-effects models typically rely upon a hierarchical generative model of the data, whereby both within- and between-subject effects contribute to the overall observed variance. The ensuing VB scheme can be used to assess statistical significance at the group level and/or to capture inter-individual differences. Alternatively, it can be seen as an adaptive regularization procedure, which iteratively learns the corresponding within-subject priors from estimates of the group distribution of effects of interest (cf. so-called “empirical bayes” approaches). We outline the mathematical derivation of the ensuing VB scheme, whose open-source implementation is available as part the VBA toolbox.
Tasks
Published	2019-03-21
URL	http://arxiv.org/abs/1903.09003v1
PDF	http://arxiv.org/pdf/1903.09003v1.pdf
PWC	https://paperswithcode.com/paper/variational-bayesian-modelling-of-mixed
Repo
Framework

AI Benchmark: All About Deep Learning on Smartphones in 2019


Title	AI Benchmark: All About Deep Learning on Smartphones in 2019
Authors	Andrey Ignatov, Radu Timofte, Andrei Kulik, Seungsoo Yang, Ke Wang, Felix Baum, Max Wu, Lirong Xu, Luc Van Gool
Abstract	The performance of mobile AI accelerators has been evolving rapidly in the past two years, nearly doubling with each new generation of SoCs. The current 4th generation of mobile NPUs is already approaching the results of CUDA-compatible Nvidia graphics cards presented not long ago, which together with the increased capabilities of mobile deep learning frameworks makes it possible to run complex and deep AI models on mobile devices. In this paper, we evaluate the performance and compare the results of all chipsets from Qualcomm, HiSilicon, Samsung, MediaTek and Unisoc that are providing hardware acceleration for AI inference. We also discuss the recent changes in the Android ML pipeline and provide an overview of the deployment of deep learning models on mobile devices. All numerical results provided in this paper can be found and are regularly updated on the official project website: http://ai-benchmark.com.
Tasks
Published	2019-10-15
URL	https://arxiv.org/abs/1910.06663v1
PDF	https://arxiv.org/pdf/1910.06663v1.pdf
PWC	https://paperswithcode.com/paper/ai-benchmark-all-about-deep-learning-on
Repo
Framework

Object Recognition with Human in the Loop Intelligent Frameworks


Title	Object Recognition with Human in the Loop Intelligent Frameworks
Authors	Orod Razeghi, Guoping Qiu
Abstract	Classifiers embedded within human in the loop visual object recognition frameworks commonly utilise two sources of information: one derived directly from the imagery data of an object, and the other obtained interactively from user interactions. These computer vision frameworks exploit human high-level cognitive power to tackle particularly difficult visual object recognition tasks. In this paper, we present innovative techniques to combine the two sources of information intelligently for the purpose of improving recognition accuracy. We firstly employ standard algorithms to build two classifiers for the two sources independently, and subsequently fuse the outputs from these classifiers to make a conclusive decision. The two fusion techniques proposed are: i) a modified naive Bayes algorithm that adaptively selects an individual classifier’s output or combines both to produce a definite answer, and ii) a neural network based algorithm which feeds the outputs of the two classifiers to a 4-layer feedforward network to generate a final output. We present extensive experimental results on 4 challenging visual recognition tasks to illustrate that the new intelligent techniques consistently outperform traditional approaches to fusing the two sources of information.
Tasks	Object Recognition
Published	2019-12-11
URL	https://arxiv.org/abs/1912.05575v1
PDF	https://arxiv.org/pdf/1912.05575v1.pdf
PWC	https://paperswithcode.com/paper/object-recognition-with-human-in-the-loop
Repo
Framework

On the Linguistic Representational Power of Neural Machine Translation Models


Title	On the Linguistic Representational Power of Neural Machine Translation Models
Authors	Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, James Glass
Abstract	Despite the recent success of deep neural networks in natural language processing (NLP), their interpretability remains a challenge. We analyze the representations learned by neural machine translation models at various levels of granularity and evaluate their quality through relevant extrinsic properties. In particular, we seek answers to the following questions: (i) How accurately is word-structure captured within the learned representations, an important aspect in translating morphologically-rich languages? (ii) Do the representations capture long-range dependencies, and effectively handle syntactically divergent languages? (iii) Do the representations capture lexical semantics? We conduct a thorough investigation along several parameters: (i) Which layers in the architecture capture each of these linguistic phenomena; (ii) How does the choice of translation unit (word, character, or subword unit) impact the linguistic properties captured by the underlying representations? (iii) Do the encoder and decoder learn differently and independently? (iv) Do the representations learned by multilingual NMT models capture the same amount of linguistic information as their bilingual counterparts? Our data-driven, quantitative evaluation illuminates important aspects in NMT models and their ability to capture various linguistic phenomena. We show that deep NMT models learn a non-trivial amount of linguistic information. Notable findings include: i) Word morphology and part-of-speech information are captured at the lower layers of the model; (ii) In contrast, lexical semantics or non-local syntactic and semantic dependencies are better represented at the higher layers; (iii) Representations learned using characters are more informed about wordmorphology compared to those learned using subword units; and (iv) Representations learned by multilingual models are richer compared to bilingual models.
Tasks	Machine Translation
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00317v1
PDF	https://arxiv.org/pdf/1911.00317v1.pdf
PWC	https://paperswithcode.com/paper/on-the-linguistic-representational-power-of
Repo
Framework

Gender Representation in French Broadcast Corpora and Its Impact on ASR Performance


Title	Gender Representation in French Broadcast Corpora and Its Impact on ASR Performance
Authors	Mahault Garnerin, Solange Rossato, Laurent Besacier
Abstract	This paper analyzes the gender representation in four major corpora of French broadcast. These corpora being widely used within the speech processing community, they are a primary material for training automatic speech recognition (ASR) systems. As gender bias has been highlighted in numerous natural language processing (NLP) applications, we study the impact of the gender imbalance in TV and radio broadcast on the performance of an ASR system. This analysis shows that women are under-represented in our data in terms of speakers and speech turns. We introduce the notion of speaker role to refine our analysis and find that women are even fewer within the Anchor category corresponding to prominent speakers. The disparity of available data for both gender causes performance to decrease on women. However this global trend can be counterbalanced for speaker who are used to speak in the media when sufficient amount of data is available.
Tasks	Speech Recognition
Published	2019-08-23
URL	https://arxiv.org/abs/1908.08717v1
PDF	https://arxiv.org/pdf/1908.08717v1.pdf
PWC	https://paperswithcode.com/paper/gender-representation-in-french-broadcast
Repo
Framework

Convergence and Margin of Adversarial Training on Separable Data


Title	Convergence and Margin of Adversarial Training on Separable Data
Authors	Zachary Charles, Shashank Rajput, Stephen Wright, Dimitris Papailiopoulos
Abstract	Adversarial training is a technique for training robust machine learning models. To encourage robustness, it iteratively computes adversarial examples for the model, and then re-trains on these examples via some update rule. This work analyzes the performance of adversarial training on linearly separable data, and provides bounds on the number of iterations required for large margin. We show that when the update rule is given by an arbitrary empirical risk minimizer, adversarial training may require exponentially many iterations to obtain large margin. However, if gradient or stochastic gradient update rules are used, only polynomially many iterations are required to find a large-margin separator. By contrast, without the use of adversarial examples, gradient methods may require exponentially many iterations to achieve large margin. Our results are derived by showing that adversarial training with gradient updates minimizes a robust version of the empirical risk at a $\mathcal{O}(\ln(t)^2/t)$ rate, despite non-smoothness. We corroborate our theory empirically.
Tasks
Published	2019-05-22
URL	https://arxiv.org/abs/1905.09209v1
PDF	https://arxiv.org/pdf/1905.09209v1.pdf
PWC	https://paperswithcode.com/paper/convergence-and-margin-of-adversarial
Repo
Framework

Transfer Learning of fMRI Dynamics


Title	Transfer Learning of fMRI Dynamics
Authors	Usman Mahmood, Md Mahfuzur Rahman, Alex Fedorov, Zening Fu, Sergey Plis
Abstract	As a mental disorder progresses, it may affect brain structure, but brain function expressed in brain dynamics is affected much earlier. Capturing the moment when brain dynamics express the disorder is crucial for early diagnosis. The traditional approach to this problem via training classifiers either proceeds from handcrafted features or requires large datasets to combat the $m»n$ problem when a high dimensional fMRI volume only has a single label that carries learning signal. Large datasets may not be available for a study of each disorder, or rare disorder types or sub-populations may not warrant for them. In this paper, we demonstrate a self-supervised pre-training method that enables us to pre-train directly on fMRI dynamics of healthy control subjects and transfer the learning to much smaller datasets of schizophrenia. Not only we enable classification of disorder directly based on fMRI dynamics in small data but also significantly speed up the learning when possible. This is encouraging evidence of informative transfer learning across datasets and diagnostic categories.
Tasks	Transfer Learning
Published	2019-11-16
URL	https://arxiv.org/abs/1911.06813v1
PDF	https://arxiv.org/pdf/1911.06813v1.pdf
PWC	https://paperswithcode.com/paper/transfer-learning-of-fmri-dynamics
Repo
Framework