Paper Group ANR 1176
Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learning. Finite Depth and Width Corrections to the Neural Tangent Kernel. Hierarchical Multi-task Deep Neural Network Architecture for End-to-End Driving. Probing Representations Learned by Multimodal Recurrent and Transformer Models. PAC Guarantees for Cooperative Multi-A …
Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learning
| Title | Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learning |
| Authors | Shariq Iqbal, Fei Sha |
| Abstract | Sparse rewards are one of the most important challenges in reinforcement learning. In the single-agent setting, these challenges have been addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces. Applying these techniques naively to the multi-agent setting results in individual agents exploring independently, without any coordination among themselves. We argue that learning in cooperative multi-agent settings can be accelerated and improved if agents coordinate with respect to what they have explored. In this paper we propose an approach for learning how to dynamically select between different types of intrinsic rewards which consider not just what an individual agent has explored, but all agents, such that the agents can coordinate their exploration and maximize extrinsic returns. Concretely, we formulate the approach as a hierarchical policy where a high-level controller selects among sets of policies trained on different types of intrinsic rewards and the low-level controllers learn the action policies of all agents under these specific rewards. We demonstrate the effectiveness of the proposed approach in a multi-agent learning domain with sparse rewards. |
| Tasks | Multi-agent Reinforcement Learning |
| Published | 2019-05-28 |
| URL | https://arxiv.org/abs/1905.12127v1 |
| https://arxiv.org/pdf/1905.12127v1.pdf | |
| PWC | https://paperswithcode.com/paper/coordinated-exploration-via-intrinsic-rewards |
| Repo | |
| Framework | |
Finite Depth and Width Corrections to the Neural Tangent Kernel
| Title | Finite Depth and Width Corrections to the Neural Tangent Kernel |
| Authors | Boris Hanin, Mihai Nica |
| Abstract | We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation is exponential in the ratio of network depth to width. Thus, even in the limit of infinite overparameterization, the NTK is not deterministic if depth and width simultaneously tend to infinity. Moreover, we prove that for such deep and wide networks, the NTK has a non-trivial evolution during training by showing that the mean of its first SGD update is also exponential in the ratio of network depth to width. This is sharp contrast to the regime where depth is fixed and network width is very large. Our results suggest that, unlike relatively shallow and wide networks, deep and wide ReLU networks are capable of learning data-dependent features even in the so-called lazy training regime. |
| Tasks | |
| Published | 2019-09-13 |
| URL | https://arxiv.org/abs/1909.05989v1 |
| https://arxiv.org/pdf/1909.05989v1.pdf | |
| PWC | https://paperswithcode.com/paper/finite-depth-and-width-corrections-to-the |
| Repo | |
| Framework | |
Hierarchical Multi-task Deep Neural Network Architecture for End-to-End Driving
| Title | Hierarchical Multi-task Deep Neural Network Architecture for End-to-End Driving |
| Authors | Jose Solomon, Francois Charette |
| Abstract | A novel hierarchical Deep Neural Network (DNN) model is presented to address the task of end-to-end driving. The model consists of a master classifier network which determines the driving task required from an input stereo image and directs said image to one of a set of subservient network regression models that perform inference and output a steering command. These subservient networks are designed and trained for a specific driving task: straightaway, swerve maneuver, tight turn, gradual turn, and chicane. Using this modular network strategy allows for two primary advantages: an overall reduction in the amount of data required to train the complete system, and for model tailoring where more complex models can be used for more challenging tasks while simplified networks can handle more mundane tasks. It is this latter facet of the model that makes the approach attractive to a number of applications beyond the current vehicle steering strategy. |
| Tasks | |
| Published | 2019-02-09 |
| URL | http://arxiv.org/abs/1902.03466v1 |
| http://arxiv.org/pdf/1902.03466v1.pdf | |
| PWC | https://paperswithcode.com/paper/hierarchical-multi-task-deep-neural-network |
| Repo | |
| Framework | |
Probing Representations Learned by Multimodal Recurrent and Transformer Models
| Title | Probing Representations Learned by Multimodal Recurrent and Transformer Models |
| Authors | Jindřich Libovický, Pranava Madhyastha |
| Abstract | Recent literature shows that large-scale language modeling provides excellent reusable sentence representations with both recurrent and self-attentive architectures. However, there has been less clarity on the commonalities and differences in the representational properties induced by the two architectures. It also has been shown that visual information serves as one of the means for grounding sentence representations. In this paper, we present a meta-study assessing the representational quality of models where the training signal is obtained from different modalities, in particular, language modeling, image features prediction, and both textual and multimodal machine translation. We evaluate textual and visual features of sentence representations obtained using predominant approaches on image retrieval and semantic textual similarity. Our experiments reveal that on moderate-sized datasets, a sentence counterpart in a target language or visual modality provides much stronger training signal for sentence representation than language modeling. Importantly, we observe that while the Transformer models achieve superior machine translation quality, representations from the recurrent neural network based models perform significantly better over tasks focused on semantic relevance. |
| Tasks | Image Retrieval, Language Modelling, Machine Translation, Multimodal Machine Translation, Semantic Textual Similarity |
| Published | 2019-08-29 |
| URL | https://arxiv.org/abs/1908.11125v1 |
| https://arxiv.org/pdf/1908.11125v1.pdf | |
| PWC | https://paperswithcode.com/paper/probing-representations-learned-by-multimodal |
| Repo | |
| Framework | |
PAC Guarantees for Cooperative Multi-Agent Reinforcement Learning with Restricted Communication
| Title | PAC Guarantees for Cooperative Multi-Agent Reinforcement Learning with Restricted Communication |
| Authors | Or Raveh, Ron Meir |
| Abstract | We develop model free PAC performance guarantees for multiple concurrent MDPs, extending recent works where a single learner interacts with multiple non-interacting agents in a noise free environment. Our framework allows noisy and resource limited communication between agents, and develops novel PAC guarantees in this extended setting. By allowing communication between the agents themselves, we suggest improved PAC-exploration algorithms that can overcome the communication noise and lead to improved sample complexity bounds. We provide a theoretically motivated algorithm that optimally combines information from the resource limited agents, thereby analyzing the interaction between noise and communication constraints that are ubiquitous in real-world systems. We present empirical results for a simple task that supports our theoretical formulations and improve upon naive information fusion methods. |
| Tasks | Multi-agent Reinforcement Learning |
| Published | 2019-05-23 |
| URL | https://arxiv.org/abs/1905.09951v2 |
| https://arxiv.org/pdf/1905.09951v2.pdf | |
| PWC | https://paperswithcode.com/paper/pac-guarantees-for-concurrent-reinforcement |
| Repo | |
| Framework | |
Adaptive Deep Learning of Cross-Domain Loss in Collaborative Filtering
| Title | Adaptive Deep Learning of Cross-Domain Loss in Collaborative Filtering |
| Authors | Dimitrios Rafailidis, Gerhard Weiss |
| Abstract | Nowadays, users open multiple accounts on social media platforms and e-commerce sites, expressing their personal preferences on different domains. However, users’ behaviors change across domains, depending on the content that users interact with, such as movies, music, clothing and retail products. In this paper, we propose an adaptive deep learning strategy for cross-domain recommendation, referred to as ADC. We design a neural architecture and formulate a cross-domain loss function, to compute the non-linearity in user preferences across domains and transfer the knowledge of users’ multiple behaviors, accordingly. In addition, we introduce an efficient algorithm for cross-domain loss balancing which directly tunes gradient magnitudes and adapts the learning rates based on the domains’ complexities/scales when training the model via backpropagation. In doing so, ADC controls and adjusts the contribution of each domain when optimizing the model parameters. Our experiments on six publicly available cross-domain recommendation tasks demonstrate the effectiveness of the proposed ADC model over other state-of-the-art methods. Furthermore, we study the effect of the proposed adaptive deep learning strategy and show that ADC can well balance the impact of the domains with different complexities. |
| Tasks | |
| Published | 2019-06-29 |
| URL | https://arxiv.org/abs/1907.01645v1 |
| https://arxiv.org/pdf/1907.01645v1.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-deep-learning-of-cross-domain-loss |
| Repo | |
| Framework | |
Visual Categorization of Objects into Animal and Plant Classes Using Global Shape Descriptors
| Title | Visual Categorization of Objects into Animal and Plant Classes Using Global Shape Descriptors |
| Authors | Zahra Sadeghi |
| Abstract | How humans can distinguish between general categories of objects? Are the subcategories of living things visually distinctive? In a number of semantic-category deficits, patients are good at making broad categorization but are unable to remember fine and specific details. It has been well accepted that general information about concepts are more robust to damages related to semantic memory. Results from patients with semantic memory disorders demonstrate the loss of ability in subcategory recognition. While bottom-up feature construction has been studied in detail, little attention has been served to top-down approach and the type of features that could account for general categorization. In this paper, we show that broad categories of animal and plant are visually distinguishable without processing textural information. To this aim, we utilize shape descriptors with an additional phase of feature learning. The results are evaluated with both supervised and unsupervised learning mechanisms. The obtained results demonstrate that global encoding of visual appearance of objects accounts for high discrimination between animal and plant object categories. |
| Tasks | |
| Published | 2019-01-25 |
| URL | http://arxiv.org/abs/1901.11398v1 |
| http://arxiv.org/pdf/1901.11398v1.pdf | |
| PWC | https://paperswithcode.com/paper/visual-categorization-of-objects-into-animal |
| Repo | |
| Framework | |
Linear Multiple Low-Rank Kernel Based Stationary Gaussian Processes Regression for Time Series
| Title | Linear Multiple Low-Rank Kernel Based Stationary Gaussian Processes Regression for Time Series |
| Authors | Feng Yin, Lishuo Pan, Xinwei He, Tianshi Chen, Sergios Theodoridis, Zhi-Quan, Luo |
| Abstract | Gaussian processes (GP) for machine learning have been studied systematically over the past two decades and they are by now widely used in a number of diverse applications. However, GP kernel design and the associated hyper-parameter optimization are still hard and to a large extend open problems. In this paper, we consider the task of GP regression for time series modeling and analysis. The underlying stationary kernel can be approximated arbitrarily close by a new proposed grid spectral mixture (GSM) kernel, which turns out to be a linear combination of low-rank sub-kernels. In the case where a large number of the sub-kernels are used, either the Nystr"{o}m or the random Fourier feature approximations can be adopted to deal efficiently with the computational demands. The unknown GP hyper-parameters consist of the non-negative weights of all sub-kernels as well as the noise variance; their estimation is performed via the maximum-likelihood (ML) estimation framework. Two efficient numerical optimization methods for solving the unknown hyper-parameters are derived, including a sequential majorization-minimization (MM) method and a non-linearly constrained alternating direction of multiplier method (ADMM). The MM matches perfectly with the proven low-rank property of the proposed GSM sub-kernels and turns out to be a part of efficiency, stable, and efficient solver, while the ADMM has the potential to generate better local minimum in terms of the test MSE. Experimental results, based on various classic time series data sets, corroborate that the proposed GSM kernel-based GP regression model outperforms several salient competitors of similar kind in terms of prediction mean-squared-error and numerical stability. |
| Tasks | Gaussian Processes, Time Series |
| Published | 2019-04-21 |
| URL | http://arxiv.org/abs/1904.09559v1 |
| http://arxiv.org/pdf/1904.09559v1.pdf | |
| PWC | https://paperswithcode.com/paper/linear-multiple-low-rank-kernel-based |
| Repo | |
| Framework | |
End-to-End Multi-Task Denoising for joint SDR and PESQ Optimization
| Title | End-to-End Multi-Task Denoising for joint SDR and PESQ Optimization |
| Authors | Jaeyoung Kim, Mostafa El-Kharmy, Jungwon Lee |
| Abstract | Supervised learning based on a deep neural network recently has achieved substantial improvement on speech enhancement. Denoising networks learn mapping from noisy speech to clean one directly, or to a spectrum mask which is the ratio between clean and noisy spectra. In either case, the network is optimized by minimizing mean square error (MSE) between ground-truth labels and time-domain or spectrum output. However, existing schemes have either of two critical issues: spectrum and metric mismatches. The spectrum mismatch is a well known issue that any spectrum modification after short-time Fourier transform (STFT), in general, cannot be fully recovered after inverse short-time Fourier transform (ISTFT). The metric mismatch is that a conventional MSE metric is sub-optimal to maximize our target metrics, signal-to-distortion ratio (SDR) and perceptual evaluation of speech quality (PESQ). This paper presents a new end-to-end denoising framework with the goal of joint SDR and PESQ optimization. First, the network optimization is performed on the time-domain signals after ISTFT to avoid spectrum mismatch. Second, two loss functions which have improved correlations with SDR and PESQ metrics are proposed to minimize metric mismatch. The experimental result showed that the proposed denoising scheme significantly improved both SDR and PESQ performance over the existing methods. |
| Tasks | Denoising, Speech Enhancement |
| Published | 2019-01-26 |
| URL | http://arxiv.org/abs/1901.09146v2 |
| http://arxiv.org/pdf/1901.09146v2.pdf | |
| PWC | https://paperswithcode.com/paper/end-to-end-multi-task-denoising-for-joint-sdr |
| Repo | |
| Framework | |
Isolation and Localization of Unknown Faults Using Neural Network-Based Residuals
| Title | Isolation and Localization of Unknown Faults Using Neural Network-Based Residuals |
| Authors | Daniel Jung |
| Abstract | Localization of unknown faults in industrial systems is a difficult task for data-driven diagnosis methods. The classification performance of many machine learning methods relies on the quality of training data. Unknown faults, for example faults not represented in training data, can be detected using, for example, anomaly classifiers. However, mapping these unknown faults to an actual location in the real system is a non-trivial problem. In model-based diagnosis, physical-based models are used to create residuals that isolate faults by mapping model equations to faulty system components. Developing sufficiently accurate physical-based models can be a time-consuming process. Hybrid modeling methods combining physical-based methods and machine learning is one solution to design data-driven residuals for fault isolation. In this work, a set of neural network-based residuals are designed by incorporating physical insights about the system behavior in the residual model structure. The residuals are trained using only fault-free data and a simulation case study shows that they can be used to perform fault isolation and localization of unknown faults in the system. |
| Tasks | |
| Published | 2019-10-12 |
| URL | https://arxiv.org/abs/1910.05626v1 |
| https://arxiv.org/pdf/1910.05626v1.pdf | |
| PWC | https://paperswithcode.com/paper/isolation-and-localization-of-unknown-faults |
| Repo | |
| Framework | |
Emergent Escape-based Flocking Behavior using Multi-Agent Reinforcement Learning
| Title | Emergent Escape-based Flocking Behavior using Multi-Agent Reinforcement Learning |
| Authors | Carsten Hahn, Thomy Phan, Thomas Gabor, Lenz Belzner, Claudia Linnhoff-Popien |
| Abstract | In nature, flocking or swarm behavior is observed in many species as it has beneficial properties like reducing the probability of being caught by a predator. In this paper, we propose SELFish (Swarm Emergent Learning Fish), an approach with multiple autonomous agents which can freely move in a continuous space with the objective to avoid being caught by a present predator. The predator has the property that it might get distracted by multiple possible preys in its vicinity. We show that this property in interaction with self-interested agents which are trained with reinforcement learning to solely survive as long as possible leads to flocking behavior similar to Boids, a common simulation for flocking behavior. Furthermore we present interesting insights in the swarming behavior and in the process of agents being caught in our modeled environment. |
| Tasks | Multi-agent Reinforcement Learning |
| Published | 2019-05-10 |
| URL | https://arxiv.org/abs/1905.04077v1 |
| https://arxiv.org/pdf/1905.04077v1.pdf | |
| PWC | https://paperswithcode.com/paper/emergent-escape-based-flocking-behavior-using |
| Repo | |
| Framework | |
Residual Encoder-Decoder Network for Deep Subspace Clustering
| Title | Residual Encoder-Decoder Network for Deep Subspace Clustering |
| Authors | Shuai Yang, Wenqi Zhu, Yuesheng Zhu |
| Abstract | Subspace clustering aims to cluster unlabeled data that lies in a union of low-dimensional linear subspaces. Deep subspace clustering approaches based on auto-encoders have become very popular to solve subspace clustering problems. However, the training of current deep methods converges slowly, which is much less efficient than traditional approaches. We propose a Residual Encoder-Decoder network for deep Subspace Clustering (RED-SC), which symmetrically links convolutional and deconvolutional layers with skip-layer connections, with which the training converges much faster. We use a self-expressive layer to generate more accurate linear representation coefficients through different latent representations from multiple latent spaces. Experiments show the superiority of RED-SC in training efficiency and clustering accuracy. Moreover, we are the first one to apply residual encoder-decoder on unsupervised learning tasks. |
| Tasks | |
| Published | 2019-10-12 |
| URL | https://arxiv.org/abs/1910.05569v1 |
| https://arxiv.org/pdf/1910.05569v1.pdf | |
| PWC | https://paperswithcode.com/paper/residual-encoder-decoder-network-for-deep |
| Repo | |
| Framework | |
Approaching Ethical Guidelines for Data Scientists
| Title | Approaching Ethical Guidelines for Data Scientists |
| Authors | Ursula Garzcarek, Detlef Steuer |
| Abstract | The goal of this article is to inspire data scientists to participate in the debate on the impact that their professional work has on society, and to become active in public debates on the digital world as data science professionals. How do ethical principles (e.g., fairness, justice, beneficence, and non-maleficence) relate to our professional lives? What lies in our responsibility as professionals by our expertise in the field? More specifically this article makes an appeal to statisticians to join that debate, and to be part of the community that establishes data science as a proper profession in the sense of Airaksinen, a philosopher working on professional ethics. As we will argue, data science has one of its roots in statistics and extends beyond it. To shape the future of statistics, and to take responsibility for the statistical contributions to data science, statisticians should actively engage in the discussions. First the term data science is defined, and the technical changes that have led to a strong influence of data science on society are outlined. Next the systematic approach from CNIL is introduced. Prominent examples are given for ethical issues arising from the work of data scientists. Further we provide reasons why data scientists should engage in shaping morality around and to formulate codes of conduct and codes of practice for data science. Next we present established ethical guidelines for the related fields of statistics and computing machinery. Thereafter necessary steps in the community to develop professional ethics for data science are described. Finally we give our starting statement for the debate: Data science is in the focal point of current societal development. Without becoming a profession with professional ethics, data science will fail in building trust in its interaction with and its much needed contributions to society! |
| Tasks | |
| Published | 2019-01-14 |
| URL | http://arxiv.org/abs/1901.04824v1 |
| http://arxiv.org/pdf/1901.04824v1.pdf | |
| PWC | https://paperswithcode.com/paper/approaching-ethical-guidelines-for-data |
| Repo | |
| Framework | |
Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs
| Title | Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs |
| Authors | Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, Bowen Zhou |
| Abstract | Multi-hop reading comprehension (RC) across documents poses new challenge over single-document RC because it requires reasoning over multiple documents to reach the final answer. In this paper, we propose a new model to tackle the multi-hop RC problem. We introduce a heterogeneous graph with different types of nodes and edges, which is named as Heterogeneous Document-Entity (HDE) graph. The advantage of HDE graph is that it contains different granularity levels of information including candidates, documents and entities in specific document contexts. Our proposed model can do reasoning over the HDE graph with nodes representation initialized with co-attention and self-attention based context encoders. We employ Graph Neural Networks (GNN) based message passing algorithms to accumulate evidences on the proposed HDE graph. Evaluated on the blind test set of the Qangaroo WikiHop data set, our HDE graph based single model delivers competitive result, and the ensemble model achieves the state-of-the-art performance. |
| Tasks | Multi-Hop Reading Comprehension, Reading Comprehension |
| Published | 2019-05-17 |
| URL | https://arxiv.org/abs/1905.07374v2 |
| https://arxiv.org/pdf/1905.07374v2.pdf | |
| PWC | https://paperswithcode.com/paper/multi-hop-reading-comprehension-across |
| Repo | |
| Framework | |
Neural Cross-Lingual Relation Extraction Based on Bilingual Word Embedding Mapping
| Title | Neural Cross-Lingual Relation Extraction Based on Bilingual Word Embedding Mapping |
| Authors | Jian Ni, Radu Florian |
| Abstract | Relation extraction (RE) seeks to detect and classify semantic relationships between entities, which provides useful information for many NLP applications. Since the state-of-the-art RE models require large amounts of manually annotated data and language-specific resources to achieve high accuracy, it is very challenging to transfer an RE model of a resource-rich language to a resource-poor language. In this paper, we propose a new approach for cross-lingual RE model transfer based on bilingual word embedding mapping. It projects word embeddings from a target language to a source language, so that a well-trained source-language neural network RE model can be directly applied to the target language. Experiment results show that the proposed approach achieves very good performance for a number of target languages on both in-house and open datasets, using a small bilingual dictionary with only 1K word pairs. |
| Tasks | Relation Extraction, Word Embeddings |
| Published | 2019-10-31 |
| URL | https://arxiv.org/abs/1911.00069v1 |
| https://arxiv.org/pdf/1911.00069v1.pdf | |
| PWC | https://paperswithcode.com/paper/neural-cross-lingual-relation-extraction-1 |
| Repo | |
| Framework | |
