Paper Group ANR 445
Learning to Discriminate Information for Online Action Detection. Self-supervised learning of class embeddings from video. Doctor2Vec: Dynamic Doctor Representation Learning for Clinical Trial Recruitment. How Much Do Unstated Problem Constraints Limit Deep Robotic Reinforcement Learning?. Variational Physics-Informed Neural Networks For Solving Pa …
Learning to Discriminate Information for Online Action Detection
| Title | Learning to Discriminate Information for Online Action Detection |
| Authors | Hyunjun Eun, Jinyoung Moon, Jongyoul Park, Chanho Jung, Changick Kim |
| Abstract | From a streaming video, online action detection aims to identify actions in the present. For this task, previous methods use recurrent networks to model the temporal sequence of current action frames. However, these methods overlook the fact that an input image sequence includes background and irrelevant actions as well as the action of interest. For online action detection, in this paper, we propose a novel recurrent unit to explicitly discriminate the information relevant to an ongoing action from others. Our unit, named Information Discrimination Unit (IDU), decides whether to accumulate input information based on its relevance to the current action. This enables our recurrent network with IDU to learn a more discriminative representation for identifying ongoing actions. In experiments on two benchmark datasets, TVSeries and THUMOS-14, the proposed method outperforms state-of-the-art methods by a significant margin. Moreover, we demonstrate the effectiveness of our recurrent unit by conducting comprehensive ablation studies. |
| Tasks | Action Detection |
| Published | 2019-12-10 |
| URL | https://arxiv.org/abs/1912.04461v3 |
| https://arxiv.org/pdf/1912.04461v3.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-discriminate-information-for |
| Repo | |
| Framework | |
Self-supervised learning of class embeddings from video
| Title | Self-supervised learning of class embeddings from video |
| Authors | Olivia Wiles, A. Sophia Koepke, Andrew Zisserman |
| Abstract | This work explores how to use self-supervised learning on videos to learn a class-specific image embedding that encodes pose and shape information. At train time, two frames of the same video of an object class (e.g. human upper body) are extracted and each encoded to an embedding. Conditioned on these embeddings, the decoder network is tasked to transform one frame into another. To successfully perform long range transformations (e.g. a wrist lowered in one image should be mapped to the same wrist raised in another), we introduce a hierarchical probabilistic network decoder model. Once trained, the embedding can be used for a variety of downstream tasks and domains. We demonstrate our approach quantitatively on three distinct deformable object classes – human full bodies, upper bodies, faces – and show experimentally that the learned embeddings do indeed generalise. They achieve state-of-the-art performance in comparison to other self-supervised methods trained on the same datasets, and approach the performance of fully supervised methods. |
| Tasks | |
| Published | 2019-10-28 |
| URL | https://arxiv.org/abs/1910.12699v1 |
| https://arxiv.org/pdf/1910.12699v1.pdf | |
| PWC | https://paperswithcode.com/paper/self-supervised-learning-of-class-embeddings |
| Repo | |
| Framework | |
Doctor2Vec: Dynamic Doctor Representation Learning for Clinical Trial Recruitment
| Title | Doctor2Vec: Dynamic Doctor Representation Learning for Clinical Trial Recruitment |
| Authors | Siddharth Biswal, Cao Xiao, Lucas M. Glass, Elizabeth Milkovits, Jimeng Sun |
| Abstract | Massive electronic health records (EHRs) enable the success of learning accurate patient representations to support various predictive health applications. In contrast, doctor representation was not well studied despite that doctors play pivotal roles in healthcare. How to construct the right doctor representations? How to use doctor representation to solve important health analytic problems? In this work, we study the problem on {\it clinical trial recruitment}, which is about identifying the right doctors to help conduct the trials based on the trial description and patient EHR data of those doctors. We propose doctor2vec which simultaneously learns 1) doctor representations from EHR data and 2) trial representations from the description and categorical information about the trials. In particular, doctor2vec utilizes a dynamic memory network where the doctor’s experience with patients are stored in the memory bank and the network will dynamically assign weights based on the trial representation via an attention mechanism. Validated on large real-world trials and EHR data including 2,609 trials, 25K doctors and 430K patients, doctor2vec demonstrated improved performance over the best baseline by up to $8.7%$ in PR-AUC. We also demonstrated that the doctor2vec embedding can be transferred to benefit data insufficiency settings including trial recruitment in less populated/newly explored country with $13.7%$ improvement or for rare diseases with $8.1%$ improvement in PR-AUC. |
| Tasks | Representation Learning |
| Published | 2019-11-23 |
| URL | https://arxiv.org/abs/1911.10395v1 |
| https://arxiv.org/pdf/1911.10395v1.pdf | |
| PWC | https://paperswithcode.com/paper/doctor2vec-dynamic-doctor-representation |
| Repo | |
| Framework | |
How Much Do Unstated Problem Constraints Limit Deep Robotic Reinforcement Learning?
| Title | How Much Do Unstated Problem Constraints Limit Deep Robotic Reinforcement Learning? |
| Authors | W. Cannon Lewis II, Mark Moll, Lydia E. Kavraki |
| Abstract | Deep Reinforcement Learning is a promising paradigm for robotic control which has been shown to be capable of learning policies for high-dimensional, continuous control of unmodeled systems. However, RoboticReinforcement Learning currently lacks clearly defined benchmark tasks, which makes it difficult for researchers to reproduce and compare against prior work. ``Reacher’’ tasks, which are fundamental to robotic manipulation, are commonly used as benchmarks, but the lack of a formal specification elides details that are crucial to replication. In this paper we present a novel empirical analysis which shows that the unstated spatial constraints in commonly used implementations of Reacher tasks make it dramatically easier to learn a successful control policy with DeepDeterministic Policy Gradients (DDPG), a state-of-the-art Deep RL algorithm. Our analysis suggests that less constrained Reacher tasks are significantly more difficult to learn, and hence that existing de facto benchmarks are not representative of the difficulty of general robotic manipulation. | |
| Tasks | Continuous Control |
| Published | 2019-09-20 |
| URL | https://arxiv.org/abs/1909.09282v1 |
| https://arxiv.org/pdf/1909.09282v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-much-do-unstated-problem-constraints |
| Repo | |
| Framework | |
Variational Physics-Informed Neural Networks For Solving Partial Differential Equations
| Title | Variational Physics-Informed Neural Networks For Solving Partial Differential Equations |
| Authors | E. Kharazmi, Z. Zhang, G. E. Karniadakis |
| Abstract | Physics-informed neural networks (PINNs) [31] use automatic differentiation to solve partial differential equations (PDEs) by penalizing the PDE in the loss function at a random set of points in the domain of interest. Here, we develop a Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neural networks (DNNs) by selecting the {\em trial space} to be the space of neural networks and the {\em test space} to be the space of Legendre polynomials. We formulate the \textit{variational residual} of the PDE using the DNN approximation by incorporating the variational form of the problem into the loss function of the network and construct a \textit{variational physics-informed neural network} (VPINN). By integrating by parts the integrand in the variational form, we lower the order of the differential operators represented by the neural networks, hence effectively reducing the training cost in VPINNs while increasing their accuracy compared to PINNs that essentially employ delta test functions. For shallow networks with one hidden layer, we analytically obtain explicit forms of the \textit{variational residual}. We demonstrate the performance of the new formulation for several examples that show clear advantages of VPINNs over PINNs in terms of both accuracy and speed. |
| Tasks | |
| Published | 2019-11-27 |
| URL | https://arxiv.org/abs/1912.00873v1 |
| https://arxiv.org/pdf/1912.00873v1.pdf | |
| PWC | https://paperswithcode.com/paper/variational-physics-informed-neural-networks |
| Repo | |
| Framework | |
Parsimonious Deep Learning: A Differential Inclusion Approach with Global Convergence
| Title | Parsimonious Deep Learning: A Differential Inclusion Approach with Global Convergence |
| Authors | Yanwei Fu, Chen Liu, Donghao Li, Xinwei Sun, Jinshan Zeng, Yuan Yao |
| Abstract | Over-parameterization is ubiquitous nowadays in training neural networks to benefit both optimization in seeking global optima and generalization in reducing prediction error. However, compressive networks are desired in many real world applications and direct training of small networks may be trapped in local optima. In this paper, instead of pruning or distilling an over-parameterized model to compressive ones, we propose a parsimonious learning approach based on differential inclusions of inverse scale spaces, that generates a family of models from simple to complex ones with a better efficiency and interpretability than stochastic gradient descent in exploring the model space. It enjoys a simple discretization, the Split Linearized Bregman Iterations, with provable global convergence that from any initializations, algorithmic iterations converge to a critical point of empirical risks. One may exploit the proposed method to boost the complexity of neural networks progressively. Numerical experiments with MNIST, Cifar-10/100, and ImageNet are conducted to show the method is promising in training large scale models with a favorite interpretability. |
| Tasks | |
| Published | 2019-05-23 |
| URL | https://arxiv.org/abs/1905.09449v2 |
| https://arxiv.org/pdf/1905.09449v2.pdf | |
| PWC | https://paperswithcode.com/paper/parsimonious-deep-learning-a-differential |
| Repo | |
| Framework | |
Attraction-Repulsion Actor-Critic for Continuous Control Reinforcement Learning
| Title | Attraction-Repulsion Actor-Critic for Continuous Control Reinforcement Learning |
| Authors | Thang Doan, Bogdan Mazoure, Audrey Durand, Joelle Pineau, R Devon Hjelm |
| Abstract | Continuous control tasks in reinforcement learning are important because they provide an important framework for learning in high-dimensional state spaces with deceptive rewards, where the agent can easily become trapped into suboptimal solutions. One way to avoid local optima is to use a population of agents to ensure coverage of the policy space, yet learning a population with the “best” coverage is still an open problem. In this work, we present a novel approach to population-based RL in continuous control that leverages properties of normalizing flows to perform attractive and repulsive operations between current members of the population and previously observed policies. Empirical results on the MuJoCo suite demonstrate a high performance gain for our algorithm compared to prior work, including Soft-Actor Critic (SAC). |
| Tasks | Continuous Control |
| Published | 2019-09-17 |
| URL | https://arxiv.org/abs/1909.07543v2 |
| https://arxiv.org/pdf/1909.07543v2.pdf | |
| PWC | https://paperswithcode.com/paper/attraction-repulsion-actor-critic-for |
| Repo | |
| Framework | |
User-Controllable Multi-Texture Synthesis with Generative Adversarial Networks
| Title | User-Controllable Multi-Texture Synthesis with Generative Adversarial Networks |
| Authors | Aibek Alanov, Max Kochurov, Denis Volkhonskiy, Daniil Yashkov, Evgeny Burnaev, Dmitry Vetrov |
| Abstract | We propose a novel multi-texture synthesis model based on generative adversarial networks (GANs) with a user-controllable mechanism. The user control ability allows to explicitly specify the texture which should be generated by the model. This property follows from using an encoder part which learns a latent representation for each texture from the dataset. To ensure a dataset coverage, we use an adversarial loss function that penalizes for incorrect reproductions of a given texture. In experiments, we show that our model can learn descriptive texture manifolds for large datasets and from raw data such as a collection of high-resolution photos. Moreover, we apply our method to produce 3D textures and show that it outperforms existing baselines. |
| Tasks | Texture Synthesis |
| Published | 2019-04-09 |
| URL | http://arxiv.org/abs/1904.04751v2 |
| http://arxiv.org/pdf/1904.04751v2.pdf | |
| PWC | https://paperswithcode.com/paper/user-controllable-multi-texture-synthesis |
| Repo | |
| Framework | |
Driving in Dense Traffic with Model-Free Reinforcement Learning
| Title | Driving in Dense Traffic with Model-Free Reinforcement Learning |
| Authors | Dhruv Mauria Saxena, Sangjae Bae, Alireza Nakhaei, Kikuo Fujimura, Maxim Likhachev |
| Abstract | Traditional planning and control methods could fail to find a feasible trajectory for an autonomous vehicle to execute amongst dense traffic on roads. This is because the obstacle-free volume in spacetime is very small in these scenarios for the vehicle to drive through. However, that does not mean the task is infeasible since human drivers are known to be able to drive amongst dense traffic by leveraging the cooperativeness of other drivers to open a gap. The traditional methods fail to take into account the fact that the actions taken by an agent affect the behaviour of other vehicles on the road. In this work, we rely on the ability of deep reinforcement learning to implicitly model such interactions and learn a continuous control policy over the action space of an autonomous vehicle. The application we consider requires our agent to negotiate and open a gap in the road in order to successfully merge or change lanes. Our policy learns to repeatedly probe into the target road lane while trying to find a safe spot to move in to. We compare against two model-predictive control-based algorithms and show that our policy outperforms them in simulation. |
| Tasks | Continuous Control |
| Published | 2019-09-15 |
| URL | https://arxiv.org/abs/1909.06710v1 |
| https://arxiv.org/pdf/1909.06710v1.pdf | |
| PWC | https://paperswithcode.com/paper/driving-in-dense-traffic-with-model-free |
| Repo | |
| Framework | |
Policy Prediction Network: Model-Free Behavior Policy with Model-Based Learning in Continuous Action Space
| Title | Policy Prediction Network: Model-Free Behavior Policy with Model-Based Learning in Continuous Action Space |
| Authors | Zac Wellmer, James Kwok |
| Abstract | This paper proposes a novel deep reinforcement learning architecture that was inspired by previous tree structured architectures which were only useable in discrete action spaces. Policy Prediction Network offers a way to improve sample complexity and performance on continuous control problems in exchange for extra computation at training time but at no cost in computation at rollout time. Our approach integrates a mix between model-free and model-based reinforcement learning. Policy Prediction Network is the first to introduce implicit model-based learning to Policy Gradient algorithms for continuous action space and is made possible via the empirically justified clipping scheme. Our experiments are focused on the MuJoCo environments so that they can be compared with similar work done in this area. |
| Tasks | Continuous Control |
| Published | 2019-09-15 |
| URL | https://arxiv.org/abs/1909.07373v1 |
| https://arxiv.org/pdf/1909.07373v1.pdf | |
| PWC | https://paperswithcode.com/paper/policy-prediction-network-model-free-behavior |
| Repo | |
| Framework | |
Neural Collaborative Subspace Clustering
| Title | Neural Collaborative Subspace Clustering |
| Authors | Tong Zhang, Pan Ji, Mehrtash Harandi, Wenbing Huang, Hongdong Li |
| Abstract | We introduce the Neural Collaborative Subspace Clustering, a neural model that discovers clusters of data points drawn from a union of low-dimensional subspaces. In contrast to previous attempts, our model runs without the aid of spectral clustering. This makes our algorithm one of the kinds that can gracefully scale to large datasets. At its heart, our neural model benefits from a classifier which determines whether a pair of points lies on the same subspace or not. Essential to our model is the construction of two affinity matrices, one from the classifier and the other from a notion of subspace self-expressiveness, to supervise training in a collaborative scheme. We thoroughly assess and contrast the performance of our model against various state-of-the-art clustering algorithms including deep subspace-based ones. |
| Tasks | |
| Published | 2019-04-24 |
| URL | http://arxiv.org/abs/1904.10596v1 |
| http://arxiv.org/pdf/1904.10596v1.pdf | |
| PWC | https://paperswithcode.com/paper/neural-collaborative-subspace-clustering |
| Repo | |
| Framework | |
From Crystallized Adaptivity to Fluid Adaptivity in Deep Reinforcement Learning – Insights from Biological Systems on Adaptive Flexibility
| Title | From Crystallized Adaptivity to Fluid Adaptivity in Deep Reinforcement Learning – Insights from Biological Systems on Adaptive Flexibility |
| Authors | Malte Schilling, Helge Ritter, Frank W. Ohl |
| Abstract | Recent developments in machine-learning algorithms have led to impressive performance increases in many traditional application scenarios of artificial intelligence research. In the area of deep reinforcement learning, deep learning functional architectures are combined with incremental learning schemes for sequential tasks that include interaction-based, but often delayed feedback. Despite their impressive successes, modern machine-learning approaches, including deep reinforcement learning, still perform weakly when compared to flexibly adaptive biological systems in certain naturally occurring scenarios. Such scenarios include transfers to environments different than the ones in which the training took place or environments that dynamically change, both of which are often mastered by biological systems through a capability that we here term “fluid adaptivity” to contrast it from the much slower adaptivity (“crystallized adaptivity”) of the prior learning from which the behavior emerged. In this article, we derive and discuss research strategies, based on analyzes of fluid adaptivity in biological systems and its neuronal modeling, that might aid in equipping future artificially intelligent systems with capabilities of fluid adaptivity more similar to those seen in some biologically intelligent systems. A key component of this research strategy is the dynamization of the problem space itself and the implementation of this dynamization by suitably designed flexibly interacting modules. |
| Tasks | |
| Published | 2019-08-13 |
| URL | https://arxiv.org/abs/1908.05348v1 |
| https://arxiv.org/pdf/1908.05348v1.pdf | |
| PWC | https://paperswithcode.com/paper/from-crystallized-adaptivity-to-fluid |
| Repo | |
| Framework | |
Learning Action-Transferable Policy with Action Embedding
| Title | Learning Action-Transferable Policy with Action Embedding |
| Authors | Yu Chen, Yingfeng Chen, Yu Yang, Ying Li, Jianwei Yin, Changjie Fan |
| Abstract | Despite achieving great success on performance in various sequential decision task, deep reinforcement learning is extremely data inefficient. Many approaches have been proposed to improve the data efficiency, e.g. transfer learning which utilizes knowledge learned from related tasks to accelerate training. Previous researches on transfer learning mostly attempt to learn a common feature space of states across related tasks to exploit knowledge as much as possible. However, semantic information of actions may be shared as well, even between tasks with different action space size. In this work, we first propose a method to learn action embedding for discrete actions in RL from generated trajectories without any prior knowledge, and then leverage it to transfer policy across tasks with different state space and/or discrete action space. We validate our method on a set of gridworld navigation tasks, discretized continuous control tasks and fighting tasks in a commercial video game. Our experimental results show that our method can effectively learn informative action embeddings and accelerate learning by policy transfer across tasks. |
| Tasks | Continuous Control, Transfer Learning |
| Published | 2019-09-05 |
| URL | https://arxiv.org/abs/1909.02291v2 |
| https://arxiv.org/pdf/1909.02291v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-action-transferable-policy-with |
| Repo | |
| Framework | |
Towards Non-I.I.D. Image Classification: A Dataset and Baselines
| Title | Towards Non-I.I.D. Image Classification: A Dataset and Baselines |
| Authors | Yue He, Zheyan Shen, Peng Cui |
| Abstract | I.I.D. hypothesis between training and testing data is the basis of numerous image classification methods. Such property can hardly be guaranteed in practice where the Non-IIDness is common, causing instable performances of these models. In literature, however, the Non-I.I.D. image classification problem is largely understudied. A key reason is lacking of a well-designed dataset to support related research. In this paper, we construct and release a Non-I.I.D. image dataset called NICO, which uses contexts to create Non-IIDness consciously. Compared to other datasets, extended analyses prove NICO can support various Non-I.I.D. situations with sufficient flexibility. Meanwhile, we propose a baseline model with ConvNet structure for General Non-I.I.D. image classification, where distribution of testing data is unknown but different from training data. The experimental results demonstrate that NICO can well support the training of ConvNet model from scratch, and a batch balancing module can help ConvNets to perform better in Non-I.I.D. settings. |
| Tasks | Image Classification |
| Published | 2019-06-07 |
| URL | https://arxiv.org/abs/1906.02899v3 |
| https://arxiv.org/pdf/1906.02899v3.pdf | |
| PWC | https://paperswithcode.com/paper/nico-a-dataset-towards-non-iid-image |
| Repo | |
| Framework | |
Single and Cross-Dimensional Feature Detection and Description: An Evaluation
| Title | Single and Cross-Dimensional Feature Detection and Description: An Evaluation |
| Authors | Odysseas Kechagias-Stamatis, Nabil Aouf, Mark A. Richardson |
| Abstract | Three-dimensional local feature detection and description techniques are widely used for object registration and recognition applications. Although several evaluations of 3D local feature detection and description methods have already been published, these are constrained in a single dimensional scheme, i.e. either 3D or 2D methods that are applied onto multiple projections of the 3D data. However, cross-dimensional (mixed 2D and 3D) feature detection and description has yet to be investigated. Here, we evaluated the performance of both single and cross-dimensional feature detection and description methods on several 3D datasets and demonstrated the superiority of cross-dimensional over single-dimensional schemes. |
| Tasks | |
| Published | 2019-10-18 |
| URL | https://arxiv.org/abs/1910.08515v1 |
| https://arxiv.org/pdf/1910.08515v1.pdf | |
| PWC | https://paperswithcode.com/paper/single-and-cross-dimensional-feature |
| Repo | |
| Framework | |
