October 20, 2019

3055 words 15 mins read

Paper Group AWR 175

Practical Deep Reinforcement Learning Approach for Stock Trading. What’s in a Domain? Learning Domain-Robust Text Representations using Adversarial Training. Visual Reinforcement Learning with Imagined Goals. An empirical study on evaluation metrics of generative adversarial networks. Exploiting Deep Representations for Neural Machine Translation. …

Practical Deep Reinforcement Learning Approach for Stock Trading


Title	Practical Deep Reinforcement Learning Approach for Stock Trading
Authors	Zhuoran Xiong, Xiao-Yang Liu, Shan Zhong, Hongyang Yang, Anwar Walid
Abstract	Stock trading strategy plays a crucial role in investment companies. However, it is challenging to obtain optimal strategy in the complex and dynamic stock market. We explore the potential of deep reinforcement learning to optimize stock trading strategy and thus maximize investment return. 30 stocks are selected as our trading stocks and their daily prices are used as the training and trading market environment. We train a deep reinforcement learning agent and obtain an adaptive trading strategy. The agent’s performance is evaluated and compared with Dow Jones Industrial Average and the traditional min-variance portfolio allocation strategy. The proposed deep reinforcement learning approach is shown to outperform the two baselines in terms of both the Sharpe ratio and cumulative returns.
Tasks
Published	2018-11-19
URL	http://arxiv.org/abs/1811.07522v2
PDF	http://arxiv.org/pdf/1811.07522v2.pdf
PWC	https://paperswithcode.com/paper/practical-deep-reinforcement-learning
Repo	https://github.com/luke-bhan/TwitterSentimentAnalysisTool
Framework	none

What’s in a Domain? Learning Domain-Robust Text Representations using Adversarial Training


Title	What’s in a Domain? Learning Domain-Robust Text Representations using Adversarial Training
Authors	Yitong Li, Timothy Baldwin, Trevor Cohn
Abstract	Most real world language problems require learning from heterogenous corpora, raising the problem of learning robust models which generalise well to both similar (in domain) and dissimilar (out of domain) instances to those seen in training. This requires learning an underlying task, while not learning irrelevant signals and biases specific to individual domains. We propose a novel method to optimise both in- and out-of-domain accuracy based on joint learning of a structured neural model with domain-specific and domain-general components, coupled with adversarial training for domain. Evaluating on multi-domain language identification and multi-domain sentiment analysis, we show substantial improvements over standard domain adaptation techniques, and domain-adversarial training.
Tasks	Domain Adaptation, Language Identification, Sentiment Analysis
Published	2018-05-16
URL	http://arxiv.org/abs/1805.06088v1
PDF	http://arxiv.org/pdf/1805.06088v1.pdf
PWC	https://paperswithcode.com/paper/whats-in-a-domain-learning-domain-robust-text
Repo	https://github.com/lrank/Domain_Robust_Text_Representation
Framework	tf

Visual Reinforcement Learning with Imagined Goals


Title	Visual Reinforcement Learning with Imagined Goals
Authors	Ashvin Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, Sergey Levine
Abstract	For an autonomous agent to fulfill a wide range of user-specified goals at test time, it must be able to learn broadly applicable and general-purpose skill repertoires. Furthermore, to provide the requisite level of generality, these skills must handle raw sensory input such as images. In this paper, we propose an algorithm that acquires such general-purpose skills by combining unsupervised representation learning and reinforcement learning of goal-conditioned policies. Since the particular goals that might be required at test-time are not known in advance, the agent performs a self-supervised “practice” phase where it imagines goals and attempts to achieve them. We learn a visual representation with three distinct purposes: sampling goals for self-supervised practice, providing a structured transformation of raw sensory inputs, and computing a reward signal for goal reaching. We also propose a retroactive goal relabeling scheme to further improve the sample-efficiency of our method. Our off-policy algorithm is efficient enough to learn policies that operate on raw image observations and goals for a real-world robotic system, and substantially outperforms prior techniques.
Tasks	Representation Learning, Unsupervised Representation Learning
Published	2018-07-12
URL	http://arxiv.org/abs/1807.04742v2
PDF	http://arxiv.org/pdf/1807.04742v2.pdf
PWC	https://paperswithcode.com/paper/visual-reinforcement-learning-with-imagined
Repo	https://github.com/vitchyr/multiworld
Framework	none

An empirical study on evaluation metrics of generative adversarial networks


Title	An empirical study on evaluation metrics of generative adversarial networks
Authors	Qiantong Xu, Gao Huang, Yang Yuan, Chuan Guo, Yu Sun, Felix Wu, Kilian Weinberger
Abstract	Evaluating generative adversarial networks (GANs) is inherently challenging. In this paper, we revisit several representative sample-based evaluation metrics for GANs, and address the problem of how to evaluate the evaluation metrics. We start with a few necessary conditions for metrics to produce meaningful scores, such as distinguishing real from generated samples, identifying mode dropping and mode collapsing, and detecting overfitting. With a series of carefully designed experiments, we comprehensively investigate existing sample-based metrics and identify their strengths and limitations in practical settings. Based on these results, we observe that kernel Maximum Mean Discrepancy (MMD) and the 1-Nearest-Neighbor (1-NN) two-sample test seem to satisfy most of the desirable properties, provided that the distances between samples are computed in a suitable feature space. Our experiments also unveil interesting properties about the behavior of several popular GAN models, such as whether they are memorizing training samples, and how far they are from learning the target distribution.
Tasks
Published	2018-06-19
URL	http://arxiv.org/abs/1806.07755v2
PDF	http://arxiv.org/pdf/1806.07755v2.pdf
PWC	https://paperswithcode.com/paper/an-empirical-study-on-evaluation-metrics-of
Repo	https://github.com/tagas/vcae
Framework	pytorch

Exploiting Deep Representations for Neural Machine Translation


Title	Exploiting Deep Representations for Neural Machine Translation
Authors	Zi-Yi Dou, Zhaopeng Tu, Xing Wang, Shuming Shi, Tong Zhang
Abstract	Advanced neural machine translation (NMT) models generally implement encoder and decoder as multiple layers, which allows systems to model complex functions and capture complicated linguistic structures. However, only the top layers of encoder and decoder are leveraged in the subsequent process, which misses the opportunity to exploit the useful information embedded in other layers. In this work, we propose to simultaneously expose all of these signals with layer aggregation and multi-layer attention mechanisms. In addition, we introduce an auxiliary regularization term to encourage different layers to capture diverse information. Experimental results on widely-used WMT14 English-German and WMT17 Chinese-English translation data demonstrate the effectiveness and universality of the proposed approach.
Tasks	Machine Translation
Published	2018-10-24
URL	http://arxiv.org/abs/1810.10181v1
PDF	http://arxiv.org/pdf/1810.10181v1.pdf
PWC	https://paperswithcode.com/paper/exploiting-deep-representations-for-neural
Repo	https://github.com/anoidgit/transformer
Framework	pytorch

Attention U-Net: Learning Where to Look for the Pancreas


Title	Attention U-Net: Learning Where to Look for the Pancreas
Authors	Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, Ben Glocker, Daniel Rueckert
Abstract	We propose a novel attention gate (AG) model for medical imaging that automatically learns to focus on target structures of varying shapes and sizes. Models trained with AGs implicitly learn to suppress irrelevant regions in an input image while highlighting salient features useful for a specific task. This enables us to eliminate the necessity of using explicit external tissue/organ localisation modules of cascaded convolutional neural networks (CNNs). AGs can be easily integrated into standard CNN architectures such as the U-Net model with minimal computational overhead while increasing the model sensitivity and prediction accuracy. The proposed Attention U-Net architecture is evaluated on two large CT abdominal datasets for multi-class image segmentation. Experimental results show that AGs consistently improve the prediction performance of U-Net across different datasets and training sizes while preserving computational efficiency. The code for the proposed architecture is publicly available.
Tasks	Medical Image Segmentation, Pancreas Segmentation, Semantic Segmentation
Published	2018-04-11
URL	http://arxiv.org/abs/1804.03999v3
PDF	http://arxiv.org/pdf/1804.03999v3.pdf
PWC	https://paperswithcode.com/paper/attention-u-net-learning-where-to-look-for
Repo	https://github.com/trichtu/Recurrent_Attention_U_net
Framework	pytorch

Identifying Recurring Patterns with Deep Neural Networks for Natural Image Denoising


Title	Identifying Recurring Patterns with Deep Neural Networks for Natural Image Denoising
Authors	Zhihao Xia, Ayan Chakrabarti
Abstract	Image denoising methods must effectively model, implicitly or explicitly, the vast diversity of patterns and textures that occur in natural images. This is challenging, even for modern methods that leverage deep neural networks trained to regress to clean images from noisy inputs. One recourse is to rely on “internal” image statistics, by searching for similar patterns within the input image itself. In this work, we propose a new method for natural image denoising that trains a deep neural network to determine whether patches in a noisy image input share common underlying patterns. Given a pair of noisy patches, our network predicts whether different sub-band coefficients of the original noise-free patches are similar. The denoising algorithm then aggregates matched coefficients to obtain an initial estimate of the clean image. Finally, this estimate is provided as input, along with the original noisy image, to a standard regression-based denoising network. Experiments show that our method achieves state-of-the-art color image denoising performance, including with a blind version that trains a common model for a range of noise levels, and does not require knowledge of level of noise in an input image. Our approach also has a distinct advantage when training with limited amounts of training data.
Tasks	Denoising, Image Denoising, Image Restoration
Published	2018-06-13
URL	https://arxiv.org/abs/1806.05229v3
PDF	https://arxiv.org/pdf/1806.05229v3.pdf
PWC	https://paperswithcode.com/paper/identifying-recurring-patterns-with-deep
Repo	https://github.com/ayanc/rpcnn
Framework	tf

Towards JointUD: Part-of-speech Tagging and Lemmatization using Recurrent Neural Networks


Title	Towards JointUD: Part-of-speech Tagging and Lemmatization using Recurrent Neural Networks
Authors	Gor Arakelyan, Karen Hambardzumyan, Hrant Khachatrian
Abstract	This paper describes our submission to CoNLL 2018 UD Shared Task. We have extended an LSTM-based neural network designed for sequence tagging to additionally generate character-level sequences. The network was jointly trained to produce lemmas, part-of-speech tags and morphological features. Sentence segmentation, tokenization and dependency parsing were handled by UDPipe 1.2 baseline. The results demonstrate the viability of the proposed multitask architecture, although its performance still remains far from state-of-the-art.
Tasks	Dependency Parsing, Lemmatization, Part-Of-Speech Tagging, Tokenization
Published	2018-09-10
URL	http://arxiv.org/abs/1809.03211v1
PDF	http://arxiv.org/pdf/1809.03211v1.pdf
PWC	https://paperswithcode.com/paper/towards-jointud-part-of-speech-tagging-and
Repo	https://github.com/Hrant-Khachatrian/Machine-Learning-in-Armenia
Framework	none

Learning to Recognize Musical Genre from Audio


Title	Learning to Recognize Musical Genre from Audio
Authors	Michaël Defferrard, Sharada P. Mohanty, Sean F. Carroll, Marcel Salathé
Abstract	We here summarize our experience running a challenge with open data for musical genre recognition. Those notes motivate the task and the challenge design, show some statistics about the submissions, and present the results.
Tasks	Music Genre Recognition
Published	2018-03-13
URL	http://arxiv.org/abs/1803.05337v1
PDF	http://arxiv.org/pdf/1803.05337v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-recognize-musical-genre-from
Repo	https://github.com/crowdAI/crowdai-musical-genre-recognition-starter-kit
Framework	none

Image Dehazing via Joint Estimation of Transmittance Map and Environmental Illumination


Title	Image Dehazing via Joint Estimation of Transmittance Map and Environmental Illumination
Authors	Sanchayan Santra, Ranjan Mondal, Pranoy Panda, Nishant Mohanty, Shubham Bhuyan
Abstract	Haze limits the visibility of outdoor images, due to the existence of fog, smoke and dust in the atmosphere. Image dehazing methods try to recover haze-free image by removing the effect of haze from a given input image. In this paper, we present an end to end system, which takes a hazy image as its input and returns a dehazed image. The proposed method learns the mapping between a hazy image and its corresponding transmittance map and the environmental illumination, by using a multi-scale Convolutional Neural Network. Although most of the time haze appears grayish in color, its color may vary depending on the color of the environmental illumination. Very few of the existing image dehazing methods have laid stress on its accurate estimation. But the color of the dehazed image and the estimated transmittance depends on the environmental illumination. Our proposed method exploits the relationship between the transmittance values and the environmental illumination as per the haze imaging model and estimates both of them. Qualitative and quantitative evaluations show, the estimates are accurate enough.
Tasks	Image Dehazing
Published	2018-12-04
URL	http://arxiv.org/abs/1812.01273v1
PDF	http://arxiv.org/pdf/1812.01273v1.pdf
PWC	https://paperswithcode.com/paper/image-dehazing-via-joint-estimation-of
Repo	https://github.com/pranoy-panda/ICAPR2017_Dehazing
Framework	tf

Character-Level Language Modeling with Deeper Self-Attention


Title	Character-Level Language Modeling with Deeper Self-Attention
Authors	Rami Al-Rfou, Dokook Choe, Noah Constant, Mandy Guo, Llion Jones
Abstract	LSTMs and other RNN variants have shown strong performance on character-level language modeling. These models are typically trained using truncated backpropagation through time, and it is common to assume that their success stems from their ability to remember long-term contexts. In this paper, we show that a deep (64-layer) transformer model with fixed context outperforms RNN variants by a large margin, achieving state of the art on two popular benchmarks: 1.13 bits per character on text8 and 1.06 on enwik8. To get good results at this depth, we show that it is important to add auxiliary losses, both at intermediate network layers and intermediate sequence positions.
Tasks	Language Modelling
Published	2018-08-09
URL	http://arxiv.org/abs/1808.04444v2
PDF	http://arxiv.org/pdf/1808.04444v2.pdf
PWC	https://paperswithcode.com/paper/character-level-language-modeling-with-deeper
Repo	https://github.com/threelittlemonkeys/transformer-pytorch
Framework	pytorch

Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog


Title	Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog
Authors	Sebastian Schuster, Sonal Gupta, Rushin Shah, Mike Lewis
Abstract	One of the first steps in the utterance interpretation pipeline of many task-oriented conversational AI systems is to identify user intents and the corresponding slots. Since data collection for machine learning models for this task is time-consuming, it is desirable to make use of existing data in a high-resource language to train models in low-resource languages. However, development of such models has largely been hindered by the lack of multilingual training data. In this paper, we present a new data set of 57k annotated utterances in English (43k), Spanish (8.6k) and Thai (5k) across the domains weather, alarm, and reminder. We use this data set to evaluate three different cross-lingual transfer methods: (1) translating the training data, (2) using cross-lingual pre-trained embeddings, and (3) a novel method of using a multilingual machine translation encoder as contextual word representations. We find that given several hundred training examples in the the target language, the latter two methods outperform translating the training data. Further, in very low-resource settings, multilingual contextual word representations give better results than using cross-lingual static embeddings. We also compare the cross-lingual methods to using monolingual resources in the form of contextual ELMo representations and find that given just small amounts of target language data, this method outperforms all cross-lingual methods, which highlights the need for more sophisticated cross-lingual methods.
Tasks	Cross-Lingual Transfer, Machine Translation, Transfer Learning
Published	2018-10-31
URL	http://arxiv.org/abs/1810.13327v2
PDF	http://arxiv.org/pdf/1810.13327v2.pdf
PWC	https://paperswithcode.com/paper/cross-lingual-transfer-learning-for
Repo	https://github.com/sz128/NLU_datasets_for_task_oriented_dialogue
Framework	pytorch


Title	Multi-Source Cross-Lingual Model Transfer: Learning What to Share
Authors	Xilun Chen, Ahmed Hassan Awadallah, Hany Hassan, Wei Wang, Claire Cardie
Abstract	Modern NLP applications have enjoyed a great boost utilizing neural networks models. Such deep neural models, however, are not applicable to most human languages due to the lack of annotated training data for various NLP tasks. Cross-lingual transfer learning (CLTL) is a viable method for building NLP models for a low-resource target language by leveraging labeled data from other (source) languages. In this work, we focus on the multilingual transfer setting where training data in multiple source languages is leveraged to further boost target language performance. Unlike most existing methods that rely only on language-invariant features for CLTL, our approach coherently utilizes both language-invariant and language-specific features at instance level. Our model leverages adversarial networks to learn language-invariant features, and mixture-of-experts models to dynamically exploit the similarity between the target language and each individual source language. This enables our model to learn effectively what to share between various languages in the multilingual setup. Moreover, when coupled with unsupervised multilingual embeddings, our model can operate in a zero-resource setting where neither target language training data nor cross-lingual resources are available. Our model achieves significant performance gains over prior art, as shown in an extensive set of experiments over multiple text classification and sequence tagging tasks including a large-scale industry dataset.
Tasks	Cross-Lingual Transfer, Text Classification, Transfer Learning
Published	2018-10-08
URL	https://arxiv.org/abs/1810.03552v3
PDF	https://arxiv.org/pdf/1810.03552v3.pdf
PWC	https://paperswithcode.com/paper/zero-resource-multilingual-model-transfer
Repo	https://github.com/microsoft/Multilingual-Model-Transfer
Framework	pytorch

Classification using Ensemble Learning under Weighted Misclassification Loss


Title	Classification using Ensemble Learning under Weighted Misclassification Loss
Authors	Yizhen Xu, Tao Liu, Michael J. Daniels, Rami Kantor, Ann Mwangi, Joseph W. Hogan
Abstract	Binary classification rules based on covariates typically depend on simple loss functions such as zero-one misclassification. Some cases may require more complex loss functions. For example, individual-level monitoring of HIV-infected individuals on antiretroviral therapy (ART) requires periodic assessment of treatment failure, defined as having a viral load (VL) value above a certain threshold. In some resource limited settings, VL tests may be limited by cost or technology, and diagnoses are based on other clinical markers. Depending on scenario, higher premium may be placed on avoiding false-positives which brings greater cost and reduced treatment options. Here, the optimal rule is determined by minimizing a weighted misclassification loss/risk. We propose a method for finding and cross-validating optimal binary classification rules under weighted misclassification loss. We focus on rules comprising a prediction score and an associated threshold, where the score is derived using an ensemble learner. Simulations and examples show that our method, which derives the score and threshold jointly, more accurately estimates overall risk and has better operating characteristics compared with methods that derive the score first and the cutoff conditionally on the score especially for finite samples.
Tasks
Published	2018-12-16
URL	https://arxiv.org/abs/1812.06507v2
PDF	https://arxiv.org/pdf/1812.06507v2.pdf
PWC	https://paperswithcode.com/paper/classification-using-ensemble-learning-under
Repo	https://github.com/yizhenxu/SL_Thresholding
Framework	none

Hydra: A Peer to Peer Distributed Training & Data Collection Framework


Title	Hydra: A Peer to Peer Distributed Training & Data Collection Framework
Authors	Vaibhav Mathur, Karanbir Chahal
Abstract	The world needs diverse and unbiased data to train deep learning models. Currently data comes from a variety of sources that are unmoderated to a large extent. The outcomes of training neural networks with unverified data yields biased models with various strains of homophobia, sexism and racism. Another trend observed in the world of deep learning is the rise of distributed training. Although cloud companies provide high performance compute for training models in the form of GPU’s connected with a low latency network, using these services comes at a high cost. We propose Hydra, a system that seeks to solve both of these problems in a novel manner by proposing a decentralized distributed framework which utilizes the substantial amount of idle compute of everyday electronic devices like smartphones and desktop computers for training and data collection purposes. Hydra couples a specialized distributed training framework on a network of these low powered devices with a reward scheme that incentivizes users to provide high quality data to unleash the compute capability on this training framework. Such a system has the ability to capture data from a wide variety of diverse sources which has been an issue in the current scenario of deep learning. Hydra brings in several new innovations in training on low powered devices including a fault tolerant version of the All Reduce algorithm. Furthermore we introduce a reinforcement learning policy to decide the size of training jobs on different machines on a heterogeneous cluster of devices with varying network latencies for Synchronous SGD. The novel thing about such a network is the ability of each machine to shut down and resume training capabilities at any point of time without restarting the overall training. To enable such an asynchronous behaviour we propose a communication framework inspired by the Bittorrent protocol and the Kademlia DHT.
Tasks
Published	2018-11-24
URL	http://arxiv.org/abs/1811.09878v1
PDF	http://arxiv.org/pdf/1811.09878v1.pdf
PWC	https://paperswithcode.com/paper/hydra-a-peer-to-peer-distributed-training
Repo	https://github.com/hydra-hoard/hydra
Framework	none