Paper Group ANR 1034
Metaoptimization on a Distributed System for Deep Reinforcement Learning. Adversarial Examples for Cost-Sensitive Classifiers. Learning Efficient Representation for Intrinsic Motivation. Self-critical n-step Training for Image Captioning. Improving Domain Adaptation Translation with Domain Invariant and Specific Information. Toward A Neuro-inspired …
Metaoptimization on a Distributed System for Deep Reinforcement Learning
| Title | Metaoptimization on a Distributed System for Deep Reinforcement Learning |
| Authors | Greg Heinrich, Iuri Frosio |
| Abstract | Training intelligent agents through reinforcement learning is a notoriously unstable procedure. Massive parallelization on GPUs and distributed systems has been exploited to generate a large amount of training experiences and consequently reduce instabilities, but the success of training remains strongly influenced by the choice of the hyperparameters. To overcome this issue, we introduce HyperTrick, a new metaoptimization algorithm, and show its effective application to tune hyperparameters in the case of deep reinforcement learning, while learning to play different Atari games on a distributed system. Our analysis provides evidence of the interaction between the identification of the optimal hyperparameters and the learned policy, that is typical of the case of metaoptimization for deep reinforcement learning. When compared with state-of-the-art metaoptimization algorithms, HyperTrick is characterized by a simpler implementation and it allows learning similar policies, while making a more effective use of the computational resources in a distributed system. |
| Tasks | Atari Games |
| Published | 2019-02-07 |
| URL | http://arxiv.org/abs/1902.02725v1 |
| http://arxiv.org/pdf/1902.02725v1.pdf | |
| PWC | https://paperswithcode.com/paper/metaoptimization-on-a-distributed-system-for |
| Repo | |
| Framework | |
Adversarial Examples for Cost-Sensitive Classifiers
| Title | Adversarial Examples for Cost-Sensitive Classifiers |
| Authors | Gavin S. Hartnett, Andrew J. Lohn, Alexander P. Sedlack |
| Abstract | Motivated by safety-critical classification problems, we investigate adversarial attacks against cost-sensitive classifiers. We use current state-of-the-art adversarially-resistant neural network classifiers [1] as the underlying models. Cost-sensitive predictions are then achieved via a final processing step in the feed-forward evaluation of the network. We evaluate the effectiveness of cost-sensitive classifiers against a variety of attacks and we introduce a new cost-sensitive attack which performs better than targeted attacks in some cases. We also explored the measures a defender can take in order to limit their vulnerability to these attacks. This attacker/defender scenario is naturally framed as a two-player zero-sum finite game which we analyze using game theory. |
| Tasks | |
| Published | 2019-10-04 |
| URL | https://arxiv.org/abs/1910.02095v1 |
| https://arxiv.org/pdf/1910.02095v1.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-examples-for-cost-sensitive |
| Repo | |
| Framework | |
Learning Efficient Representation for Intrinsic Motivation
| Title | Learning Efficient Representation for Intrinsic Motivation |
| Authors | Ruihan Zhao, Stas Tiomkin, Pieter Abbeel |
| Abstract | Mutual Information between agent Actions and environment States (MIAS) quantifies the influence of agent on its environment. Recently, it was found that the maximization of MIAS can be used as an intrinsic motivation for artificial agents. In literature, the term empowerment is used to represent the maximum of MIAS at a certain state. While empowerment has been shown to solve a broad range of reinforcement learning problems, its calculation in arbitrary dynamics is a challenging problem because it relies on the estimation of mutual information. Existing approaches, which rely on sampling, are limited to low dimensional spaces, because high-confidence distribution-free lower bounds for mutual information require exponential number of samples. In this work, we develop a novel approach for the estimation of empowerment in unknown dynamics from visual observation only, without the need to sample for MIAS. The core idea is to represent the relation between action sequences and future states using a stochastic dynamic model in latent space with a specific form. This allows us to efficiently compute empowerment with the “Water-Filling” algorithm from information theory. We construct this embedding with deep neural networks trained on a sophisticated objective function. Our experimental results show that the designed embedding preserves information-theoretic properties of the original dynamics. |
| Tasks | |
| Published | 2019-12-04 |
| URL | https://arxiv.org/abs/1912.02624v2 |
| https://arxiv.org/pdf/1912.02624v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-efficient-representation-for |
| Repo | |
| Framework | |
Self-critical n-step Training for Image Captioning
| Title | Self-critical n-step Training for Image Captioning |
| Authors | Junlong Gao, Shiqi Wang, Shanshe Wang, Siwei Ma, Wen Gao |
| Abstract | Existing methods for image captioning are usually trained by cross entropy loss, which leads to exposure bias and the inconsistency between the optimizing function and evaluation metrics. Recently it has been shown that these two issues can be addressed by incorporating techniques from reinforcement learning, where one of the popular techniques is the advantage actor-critic algorithm that calculates per-token advantage by estimating state value with a parametrized estimator at the cost of introducing estimation bias. In this paper, we estimate state value without using a parametrized value estimator. With the properties of image captioning, namely, the deterministic state transition function and the sparse reward, state value is equivalent to its preceding state-action value, and we reformulate advantage function by simply replacing the former with the latter. Moreover, the reformulated advantage is extended to n-step, which can generally increase the absolute value of the mean of reformulated advantage while lowering variance. Then two kinds of rollout are adopted to estimate state-action value, which we call self-critical n-step training. Empirically we find that our method can obtain better performance compared to the state-of-the-art methods that use the sequence level advantage and parametrized estimator respectively on the widely used MSCOCO benchmark. |
| Tasks | Image Captioning |
| Published | 2019-04-15 |
| URL | http://arxiv.org/abs/1904.06861v1 |
| http://arxiv.org/pdf/1904.06861v1.pdf | |
| PWC | https://paperswithcode.com/paper/self-critical-n-step-training-for-image |
| Repo | |
| Framework | |
Improving Domain Adaptation Translation with Domain Invariant and Specific Information
| Title | Improving Domain Adaptation Translation with Domain Invariant and Specific Information |
| Authors | Shuhao Gu, Yang Feng, Qun Liu |
| Abstract | In domain adaptation for neural machine translation, translation performance can benefit from separating features into domain-specific features and common features. In this paper, we propose a method to explicitly model the two kinds of information in the encoder-decoder framework so as to exploit out-of-domain data in in-domain training. In our method, we maintain a private encoder and a private decoder for each domain which are used to model domain-specific information. In the meantime, we introduce a common encoder and a common decoder shared by all the domains which can only have domain-independent information flow through. Besides, we add a discriminator to the shared encoder and employ adversarial training for the whole model to reinforce the performance of information separation and machine translation simultaneously. Experiment results show that our method can outperform competitive baselines greatly on multiple data sets. |
| Tasks | Domain Adaptation, Machine Translation |
| Published | 2019-04-08 |
| URL | https://arxiv.org/abs/1904.03879v2 |
| https://arxiv.org/pdf/1904.03879v2.pdf | |
| PWC | https://paperswithcode.com/paper/improving-domain-adaptation-translation-with |
| Repo | |
| Framework | |
Toward A Neuro-inspired Creative Decoder
| Title | Toward A Neuro-inspired Creative Decoder |
| Authors | Payel Das, Brian Quanz, Pin-Yu Chen, Jae-wook Ahn |
| Abstract | Creativity, a process that generates novel and valuable ideas, involves increased association between task-positive (control) and task-negative (default) networks in brain. Inspired by this seminal finding, in this study we propose a creative decoder that directly modulates the neuronal activation pattern, while sampling from the learned latent space. The proposed approach is fully unsupervised and can be used as off-the-shelf. Our experiments on three different image datasets (MNIST, FMNIST, CELEBA) reveal that the co-activation between task-positive and task-negative neurons during decoding in a deep neural net enables generation of novel artifacts. We further identify sufficient conditions on several novelty metrics towards measuring the creativity of generated samples. |
| Tasks | |
| Published | 2019-02-06 |
| URL | http://arxiv.org/abs/1902.02399v2 |
| http://arxiv.org/pdf/1902.02399v2.pdf | |
| PWC | https://paperswithcode.com/paper/toward-a-neuro-inspired-creative-decoder |
| Repo | |
| Framework | |
End-to-end Training of CNN-CRF via Differentiable Dual-Decomposition
| Title | End-to-end Training of CNN-CRF via Differentiable Dual-Decomposition |
| Authors | Shaofei Wang, Vishnu Lokhande, Maneesh Singh, Konrad Kording, Julian Yarkony |
| Abstract | Modern computer vision (CV) is often based on convolutional neural networks (CNNs) that excel at hierarchical feature extraction. The previous generation of CV approaches was often based on conditional random fields (CRFs) that excel at modeling flexible higher order interactions. As their benefits are complementary they are often combined. However, these approaches generally use mean-field approximations and thus, arguably, did not directly optimize the real problem. Here we revisit dual-decomposition-based approaches to CRF optimization, an alternative to the mean-field approximation. These algorithms can efficiently and exactly solve sub-problems and directly optimize a convex upper bound of the real problem, providing optimality certificates on the way. Our approach uses a novel fixed-point iteration algorithm which enjoys dual-monotonicity, dual-differentiability and high parallelism. The whole system, CRF and CNN can thus be efficiently trained using back-propagation. We demonstrate the effectiveness of our system on semantic image segmentation, showing consistent improvement over baseline models. |
| Tasks | Semantic Segmentation |
| Published | 2019-12-06 |
| URL | https://arxiv.org/abs/1912.02937v1 |
| https://arxiv.org/pdf/1912.02937v1.pdf | |
| PWC | https://paperswithcode.com/paper/end-to-end-training-of-cnn-crf-via |
| Repo | |
| Framework | |
Sensing Social Media Signals for Cryptocurrency News
| Title | Sensing Social Media Signals for Cryptocurrency News |
| Authors | Johannes Beck, Roberta Huang, David Lindner, Tian Guo, Ce Zhang, Dirk Helbing, Nino Antulov-Fantulin |
| Abstract | The ability to track and monitor relevant and important news in real-time is of crucial interest in multiple industrial sectors. In this work, we focus on the set of cryptocurrency news, which recently became of emerging interest to the general and financial audience. In order to track relevant news in real-time, we (i) match news from the web with tweets from social media, (ii) track their intraday tweet activity and (iii) explore different machine learning models for predicting the number of the article mentions on Twitter within the first 24 hours after its publication. We compare several machine learning models, such as linear extrapolation, linear and random forest autoregressive models, and a sequence-to-sequence neural network. We find that the random forest autoregressive model behaves comparably to more complex models in the majority of tasks. |
| Tasks | |
| Published | 2019-03-27 |
| URL | http://arxiv.org/abs/1903.11451v1 |
| http://arxiv.org/pdf/1903.11451v1.pdf | |
| PWC | https://paperswithcode.com/paper/sensing-social-media-signals-for |
| Repo | |
| Framework | |
Large Margin Multi-modal Multi-task Feature Extraction for Image Classification
| Title | Large Margin Multi-modal Multi-task Feature Extraction for Image Classification |
| Authors | Yong Luo, Yonggang Wen, Dacheng Tao, Jie Gui, Chao Xu |
| Abstract | The features used in many image analysis-based applications are frequently of very high dimension. Feature extraction offers several advantages in high-dimensional cases, and many recent studies have used multi-task feature extraction approaches, which often outperform single-task feature extraction approaches. However, most of these methods are limited in that they only consider data represented by a single type of feature, even though features usually represent images from multiple modalities. We therefore propose a novel large margin multi-modal multi-task feature extraction (LM3FE) framework for handling multi-modal features for image classification. In particular, LM3FE simultaneously learns the feature extraction matrix for each modality and the modality combination coefficients. In this way, LM3FE not only handles correlated and noisy features, but also utilizes the complementarity of different modalities to further help reduce feature redundancy in each modality. The large margin principle employed also helps to extract strongly predictive features so that they are more suitable for prediction (e.g., classification). An alternating algorithm is developed for problem optimization and each sub-problem can be efficiently solved. Experiments on two challenging real-world image datasets demonstrate the effectiveness and superiority of the proposed method. |
| Tasks | Image Classification |
| Published | 2019-04-08 |
| URL | http://arxiv.org/abs/1904.04088v1 |
| http://arxiv.org/pdf/1904.04088v1.pdf | |
| PWC | https://paperswithcode.com/paper/large-margin-multi-modal-multi-task-feature |
| Repo | |
| Framework | |
Efficient Inference and Exploration for Reinforcement Learning
| Title | Efficient Inference and Exploration for Reinforcement Learning |
| Authors | YI Zhu, Jing Dong, Henry Lam |
| Abstract | Despite an ever growing literature on reinforcement learning algorithms and applications, much less is known about their statistical inference. In this paper, we investigate the large sample behaviors of the Q-value estimates with closed-form characterizations of the asymptotic variances. This allows us to efficiently construct confidence regions for Q-value and optimal value functions, and to develop policies to minimize their estimation errors. This also leads to a policy exploration strategy that relies on estimating the relative discrepancies among the Q estimates. Numerical experiments show superior performances of our exploration strategy than other benchmark approaches. |
| Tasks | |
| Published | 2019-10-12 |
| URL | https://arxiv.org/abs/1910.05471v2 |
| https://arxiv.org/pdf/1910.05471v2.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-inference-and-exploration-for |
| Repo | |
| Framework | |
neuralRank: Searching and ranking ANN-based model repositories
| Title | neuralRank: Searching and ranking ANN-based model repositories |
| Authors | Nirmit Desai, Linsong Chu, Raghu K. Ganti, Sebastian Stein, Mudhakar Srivatsa |
| Abstract | Widespread applications of deep learning have led to a plethora of pre-trained neural network models for common tasks. Such models are often adapted from other models via transfer learning. The models may have varying training sets, training algorithms, network architectures, and hyper-parameters. For a given application, what isthe most suitable model in a model repository? This is a critical question for practical deployments but it has not received much attention. This paper introduces the novel problem of searching and ranking models based on suitability relative to a target dataset and proposes a ranking algorithm called \textit{neuralRank}. The key idea behind this algorithm is to base model suitability on the discriminating power of a model, using a novel metric to measure it. With experimental results on the MNIST, Fashion, and CIFAR10 datasets, we demonstrate that (1) neuralRank is independent of the domain, the training set, or the network architecture and (2) that the models ranked highly by neuralRank ranking tend to have higher model accuracy in practice. |
| Tasks | Transfer Learning |
| Published | 2019-03-02 |
| URL | http://arxiv.org/abs/1903.00711v1 |
| http://arxiv.org/pdf/1903.00711v1.pdf | |
| PWC | https://paperswithcode.com/paper/neuralrank-searching-and-ranking-ann-based |
| Repo | |
| Framework | |
Align-and-Attend Network for Globally and Locally Coherent Video Inpainting
| Title | Align-and-Attend Network for Globally and Locally Coherent Video Inpainting |
| Authors | Sanghyun Woo, Dahun Kim, KwanYong Park, Joon-Young Lee, In So Kweon |
| Abstract | We propose a novel feed-forward network for video inpainting. We use a set of sampled video frames as the reference to take visible contents to fill the hole of a target frame. Our video inpainting network consists of two stages. The first stage is an alignment module that uses computed homographies between the reference frames and the target frame. The visible patches are then aggregated based on the frame similarity to fill in the target holes roughly. The second stage is a non-local attention module that matches the generated patches with known reference patches (in space and time) to refine the previous global alignment stage. Both stages consist of large spatial-temporal window size for the reference and thus enable modeling long-range correlations between distant information and the hole regions. Therefore, even challenging scenes with large or slowly moving holes can be handled, which have been hardly modeled by existing flow-based approach. Our network is also designed with a recurrent propagation stream to encourage temporal consistency in video results. Experiments on video object removal demonstrate that our method inpaints the holes with globally and locally coherent contents. |
| Tasks | Video Inpainting |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1905.13066v1 |
| https://arxiv.org/pdf/1905.13066v1.pdf | |
| PWC | https://paperswithcode.com/paper/align-and-attend-network-for-globally-and |
| Repo | |
| Framework | |
Unifying Question Answering, Text Classification, and Regression via Span Extraction
| Title | Unifying Question Answering, Text Classification, and Regression via Span Extraction |
| Authors | Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, Richard Socher |
| Abstract | Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering, text classification, and regression models are significantly different. Span decoders are frequently used for question answering, fixed-class, classification layers for text classification, and similarity-scoring layers for regression tasks, We show that this distinction is not necessary and that all three can be unified as span extraction. A unified, span-extraction approach leads to superior or comparable performance in supplementary supervised pre-trained, low-data, and multi-task learning experiments on several question answering, text classification, and regression benchmarks. |
| Tasks | Multi-Task Learning, Question Answering, Text Classification |
| Published | 2019-04-19 |
| URL | https://arxiv.org/abs/1904.09286v2 |
| https://arxiv.org/pdf/1904.09286v2.pdf | |
| PWC | https://paperswithcode.com/paper/unifying-question-answering-and-text |
| Repo | |
| Framework | |
The phonetic bases of vocal expressed emotion: natural versus acted
| Title | The phonetic bases of vocal expressed emotion: natural versus acted |
| Authors | Hira Dhamyal, Shahan A. Memon, Bhiksha Raj, Rita Singh |
| Abstract | Can vocal emotions be emulated? This question has been a recurrent concern of the speech community, and has also been vigorously investigated. It has been fueled further by its link to the issue of validity of acted emotion databases. Much of the speech and vocal emotion research has relied on acted emotion databases as valid proxies for studying natural emotions. To create models that generalize to natural settings, it is crucial to work with valid prototypes – ones that can be assumed to reliably represent natural emotions. More concretely, it is important to study emulated emotions against natural emotions in terms of their physiological, and psychological concomitants. In this paper, we present an on-scale systematic study of the differences between natural and acted vocal emotions. We use a self-attention based emotion classification model to understand the phonetic bases of emotions by discovering the most attentive phonemes for each class of emotions. We then compare these attentive phonemes in their importance and distribution across acted and natural classes. Our conclusions show significant differences in the manner and choice of phonemes in acted and natural speech, concluding moderate to low validity and value in using acted speech databases for emotion classification tasks. |
| Tasks | Emotion Classification |
| Published | 2019-11-13 |
| URL | https://arxiv.org/abs/1911.05733v2 |
| https://arxiv.org/pdf/1911.05733v2.pdf | |
| PWC | https://paperswithcode.com/paper/the-phonetic-bases-of-vocal-expressed-emotion |
| Repo | |
| Framework | |
Deep Compressed Pneumonia Detection for Low-Power Embedded Devices
| Title | Deep Compressed Pneumonia Detection for Low-Power Embedded Devices |
| Authors | Hongjia Li, Sheng Lin, Ning Liu, Caiwen Ding, Yanzhi Wang |
| Abstract | Deep neural networks (DNNs) have been expanded into medical fields and triggered the revolution of some medical applications by extracting complex features and achieving high accuracy and performance, etc. On the contrast, the large-scale network brings high requirements of both memory storage and computation resource, especially for portable medical devices and other embedded systems. In this work, we first train a DNN for pneumonia detection using the dataset provided by RSNA Pneumonia Detection Challenge. To overcome hardware limitation for implementing large-scale networks, we develop a systematic structured weight pruning method with filter sparsity, column sparsity and combined sparsity. Experiments show that we can achieve up to 36x compression ratio compared to the original model with 106 layers, while maintaining no accuracy degradation. We evaluate the proposed methods on an embedded low-power device, Jetson TX2, and achieve low power usage and high energy efficiency. |
| Tasks | Pneumonia Detection |
| Published | 2019-11-04 |
| URL | https://arxiv.org/abs/1911.02007v1 |
| https://arxiv.org/pdf/1911.02007v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-compressed-pneumonia-detection-for-low |
| Repo | |
| Framework | |
