Paper Group ANR 1723
Leveraging Reinforcement Learning Techniques for Effective Policy Adoption and Validation. Mining Human Mobility Data to Discover Locations and Habits. How robots in a large group make decisions as a whole? From biological inspiration to the design of distributed algorithms. Software Engineering Meets Deep Learning: A Literature Review. Convergence …
Leveraging Reinforcement Learning Techniques for Effective Policy Adoption and Validation
| Title | Leveraging Reinforcement Learning Techniques for Effective Policy Adoption and Validation |
| Authors | Nikki Lijing Kuang, Clement H. C. Leung |
| Abstract | Rewards and punishments in different forms are pervasive and present in a wide variety of decision-making scenarios. By observing the outcome of a sufficient number of repeated trials, one would gradually learn the value and usefulness of a particular policy or strategy. However, in a given environment, the outcomes resulting from different trials are subject to chance influence and variations. In learning about the usefulness of a given policy, significant costs are involved in systematically undertaking the sequential trials; therefore, in most learning episodes, one would wish to keep the cost within bounds by adopting learning stopping rules. In this paper, we examine the deployment of different stopping strategies in given learning environments which vary from highly stringent for mission critical operations to highly tolerant for non-mission critical operations, and emphasis is placed on the former with particular application to aviation safety. In policy evaluation, two sequential phases of learning are identified, and we describe the outcomes variations using a probabilistic model, with closedform expressions obtained for the key measures of performance. Decision rules that map the trial observations to policy choices are also formulated. In addition, simulation experiments are performed, which corroborate the validity of the theoretical results. |
| Tasks | Decision Making |
| Published | 2019-06-21 |
| URL | https://arxiv.org/abs/1906.09340v1 |
| https://arxiv.org/pdf/1906.09340v1.pdf | |
| PWC | https://paperswithcode.com/paper/leveraging-reinforcement-learning-techniques |
| Repo | |
| Framework | |
Mining Human Mobility Data to Discover Locations and Habits
| Title | Mining Human Mobility Data to Discover Locations and Habits |
| Authors | Thiago Andrade, Brais Cancela, João Gama |
| Abstract | Many aspects of life are associated with places of human mobility patterns and nowadays we are facing an increase in the pervasiveness of mobile devices these individuals carry. Positioning technologies that serve these devices such as the cellular antenna (GSM networks), global navigation satellite systems (GPS), and more recently the WiFi positioning system (WPS) provide large amounts of spatio-temporal data in a continuous way. Therefore, detecting significant places and the frequency of movements between them is fundamental to understand human behavior. In this paper, we propose a method for discovering user habits without any a priori or external knowledge by introducing a density-based clustering for spatio-temporal data to identify meaningful places and by applying a Gaussian Mixture Model (GMM) over the set of meaningful places to identify the representations of individual habits. To evaluate the proposed method we use two real-world datasets. One dataset contains high-density GPS data and the other one contains GSM mobile phone data in a coarse representation. The results show that the proposed method is suitable for this task as many unique habits were identified. This can be used for understanding users’ behavior and to draw their characterizing profiles having a panorama of the mobility patterns from the data. |
| Tasks | |
| Published | 2019-09-25 |
| URL | https://arxiv.org/abs/1909.11406v1 |
| https://arxiv.org/pdf/1909.11406v1.pdf | |
| PWC | https://paperswithcode.com/paper/mining-human-mobility-data-to-discover |
| Repo | |
| Framework | |
How robots in a large group make decisions as a whole? From biological inspiration to the design of distributed algorithms
| Title | How robots in a large group make decisions as a whole? From biological inspiration to the design of distributed algorithms |
| Authors | Gabriele Valentini |
| Abstract | Nature provides us with abundant examples of how large numbers of individuals can make decisions without the coordination of a central authority. Social insects, birds, fishes, and many other living collectives, rely on simple interaction mechanisms to do so. They individually gather information from the environment; small bits of a much larger picture that are then shared locally among the members of the collective and processed together to output a commonly agreed choice. Throughout evolution, Nature found solutions to collective decision-making problems that are intriguing to engineers for their robustness to malfunctioning or lost individuals, their flexibility in face of dynamic environments, and their ability to scale with large numbers of members. In the last decades, whereas biologists amassed large amounts of experimental evidence, engineers took inspiration from these and other examples to design distributed algorithms that, while maintaining the same properties of their natural counterparts, come with guarantees on their performance in the form of predictive mathematical models. In this paper, we review the fundamental processes that lead to a collective decision. We discuss examples of collective decisions in biological systems and show how similar processes can be engineered to design artificial ones. During this journey, we review a framework to design distributed decision-making algorithms that are modular, can be instantiated and extended in different ways, and are supported by a suit of predictive mathematical models. |
| Tasks | Decision Making |
| Published | 2019-10-24 |
| URL | https://arxiv.org/abs/1910.11262v2 |
| https://arxiv.org/pdf/1910.11262v2.pdf | |
| PWC | https://paperswithcode.com/paper/how-robots-in-a-large-group-make-decisions-as |
| Repo | |
| Framework | |
Software Engineering Meets Deep Learning: A Literature Review
| Title | Software Engineering Meets Deep Learning: A Literature Review |
| Authors | Fabio Ferreira, Luciana Lourdes Silva, Marco Tulio Valente |
| Abstract | Deep learning (DL) is being used nowadays in many traditional software engineering (SE) problems and tasks, such as software documentation, defect prediction, and software testing. However, since the renaissance of DL techniques is still very recent, we lack works that summarize and condense the most recent and relevant research conducted in the intersection of DL and SE. Therefore, in this paper we describe the first results of a literature review covering 81 papers about DL & SE. |
| Tasks | |
| Published | 2019-09-25 |
| URL | https://arxiv.org/abs/1909.11436v2 |
| https://arxiv.org/pdf/1909.11436v2.pdf | |
| PWC | https://paperswithcode.com/paper/software-engineering-meets-deep-learning-a |
| Repo | |
| Framework | |
Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Data
| Title | Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Data |
| Authors | Shicong Cen, Huishuai Zhang, Yuejie Chi, Wei Chen, Tie-Yan Liu |
| Abstract | Stochastic variance reduced methods have gained a lot of interest recently for empirical risk minimization due to its appealing run time complexity. When the data size is large and disjointly stored on different machines, it becomes imperative to distribute the implementation of such variance reduced methods. In this paper, we consider a general framework that directly distributes popular stochastic variance reduced methods, by assigning outer loops to the parameter server, and inner loops to worker machines. This framework is natural as it does not require sampling extra data and is friendly to implement, but its theoretical convergence is not well understood. We obtain a unified understanding of the convergence for algorithms under this framework by measuring the smoothness of the discrepancy between the local and global loss functions. We establish the linear convergence of distributed versions of a family of stochastic variance reduced algorithms, including those using accelerated and recursive gradient updates, for minimizing strongly convex losses. Our theory captures how the convergence of distributed algorithms behaves as the number of machines and the size of local data vary. Furthermore, we show that when the smoothness discrepancy between local and global loss functions is large, regularization can be used to ensure convergence. Our analysis can be further extended to handle nonsmooth and nonconvex loss functions. |
| Tasks | |
| Published | 2019-05-29 |
| URL | https://arxiv.org/abs/1905.12648v2 |
| https://arxiv.org/pdf/1905.12648v2.pdf | |
| PWC | https://paperswithcode.com/paper/convergence-of-distributed-stochastic |
| Repo | |
| Framework | |
Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrieval
| Title | Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrieval |
| Authors | Sijin Wang, Ruiping Wang, Ziwei Yao, Shiguang Shan, Xilin Chen |
| Abstract | Image-text retrieval of natural scenes has been a popular research topic. Since image and text are heterogeneous cross-modal data, one of the key challenges is how to learn comprehensive yet unified representations to express the multi-modal data. A natural scene image mainly involves two kinds of visual concepts, objects and their relationships, which are equally essential to image-text retrieval. Therefore, a good representation should account for both of them. In the light of recent success of scene graph in many CV and NLP tasks for describing complex natural scenes, we propose to represent image and text with two kinds of scene graphs: visual scene graph (VSG) and textual scene graph (TSG), each of which is exploited to jointly characterize objects and relationships in the corresponding modality. The image-text retrieval task is then naturally formulated as cross-modal scene graph matching. Specifically, we design two particular scene graph encoders in our model for VSG and TSG, which can refine the representation of each node on the graph by aggregating neighborhood information. As a result, both object-level and relationship-level cross-modal features can be obtained, which favorably enables us to evaluate the similarity of image and text in the two levels in a more plausible way. We achieve state-of-the-art results on Flickr30k and MSCOCO, which verifies the advantages of our graph matching based approach for image-text retrieval. |
| Tasks | Graph Matching |
| Published | 2019-10-11 |
| URL | https://arxiv.org/abs/1910.05134v1 |
| https://arxiv.org/pdf/1910.05134v1.pdf | |
| PWC | https://paperswithcode.com/paper/cross-modal-scene-graph-matching-for |
| Repo | |
| Framework | |
Scaling Autoregressive Video Models
| Title | Scaling Autoregressive Video Models |
| Authors | Dirk Weissenborn, Oscar Täckström, Jakob Uszkoreit |
| Abstract | Due to the statistical complexity of video, the high degree of inherent stochasticity, and the sheer amount of data, generating natural video remains a challenging task. State-of-the-art video generation models often attempt to address these issues by combining sometimes complex, usually video-specific neural network architectures, latent variable models, adversarial training and a range of other methods. Despite their often high complexity, these approaches still fall short of generating high quality video continuations outside of narrow domains and often struggle with fidelity. In contrast, we show that conceptually simple autoregressive video generation models based on a three-dimensional self-attention mechanism achieve competitive results across multiple metrics on popular benchmark datasets, for which they produce continuations of high fidelity and realism. We also present results from training our models on Kinetics, a large scale action recognition dataset comprised of YouTube videos exhibiting phenomena such as camera movement, complex object interactions and diverse human movement. While modeling these phenomena consistently remains elusive, we hope that our results, which include occasional realistic continuations encourage further research on comparatively complex, large scale datasets such as Kinetics. |
| Tasks | Latent Variable Models, Video Generation |
| Published | 2019-06-06 |
| URL | https://arxiv.org/abs/1906.02634v3 |
| https://arxiv.org/pdf/1906.02634v3.pdf | |
| PWC | https://paperswithcode.com/paper/scaling-autoregressive-video-models |
| Repo | |
| Framework | |
Spatio-Temporal Vegetation Pixel Classification By Using Convolutional Networks
| Title | Spatio-Temporal Vegetation Pixel Classification By Using Convolutional Networks |
| Authors | Keiller Nogueira, Jefersson A. dos Santos, Nathalia Menini, Thiago S. F. Silva, Leonor Patricia C. Morellato, Ricardo da S. Torres |
| Abstract | Plant phenology studies rely on long-term monitoring of life cycles of plants. High-resolution unmanned aerial vehicles (UAVs) and near-surface technologies have been used for plant monitoring, demanding the creation of methods capable of locating and identifying plant species through time and space. However, this is a challenging task given the high volume of data, the constant data missing from temporal dataset, the heterogeneity of temporal profiles, the variety of plant visual patterns, and the unclear definition of individuals’ boundaries in plant communities. In this letter, we propose a novel method, suitable for phenological monitoring, based on Convolutional Networks (ConvNets) to perform spatio-temporal vegetation pixel-classification on high resolution images. We conducted a systematic evaluation using high-resolution vegetation image datasets associated with the Brazilian Cerrado biome. Experimental results show that the proposed approach is effective, overcoming other spatio-temporal pixel-classification strategies. |
| Tasks | |
| Published | 2019-03-02 |
| URL | http://arxiv.org/abs/1903.00774v1 |
| http://arxiv.org/pdf/1903.00774v1.pdf | |
| PWC | https://paperswithcode.com/paper/spatio-temporal-vegetation-pixel |
| Repo | |
| Framework | |
Meta Reinforcement Learning for Sim-to-real Domain Adaptation
| Title | Meta Reinforcement Learning for Sim-to-real Domain Adaptation |
| Authors | Karol Arndt, Murtaza Hazara, Ali Ghadirzadeh, Ville Kyrki |
| Abstract | Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance. |
| Tasks | Domain Adaptation, Meta-Learning |
| Published | 2019-09-16 |
| URL | https://arxiv.org/abs/1909.12906v1 |
| https://arxiv.org/pdf/1909.12906v1.pdf | |
| PWC | https://paperswithcode.com/paper/meta-reinforcement-learning-for-sim-to-real |
| Repo | |
| Framework | |
Text-based Editing of Talking-head Video
| Title | Text-based Editing of Talking-head Video |
| Authors | Ohad Fried, Ayush Tewari, Michael Zollhöfer, Adam Finkelstein, Eli Shechtman, Dan B Goldman, Kyle Genova, Zeyu Jin, Christian Theobalt, Maneesh Agrawala |
| Abstract | Editing talking-head video to change the speech content or to remove filler words is challenging. We propose a novel method to edit talking-head video based on its transcript to produce a realistic output video in which the dialogue of the speaker has been modified, while maintaining a seamless audio-visual flow (i.e. no jump cuts). Our method automatically annotates an input talking-head video with phonemes, visemes, 3D face pose and geometry, reflectance, expression and scene illumination per frame. To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material. The annotated parameters corresponding to the selected segments are seamlessly stitched together and used to produce an intermediate video representation in which the lower half of the face is rendered with a parametric face model. Finally, a recurrent video generation network transforms this representation to a photorealistic video that matches the edited transcript. We demonstrate a large variety of edits, such as the addition, removal, and alteration of words, as well as convincing language translation and full sentence synthesis. |
| Tasks | Video Generation |
| Published | 2019-06-04 |
| URL | https://arxiv.org/abs/1906.01524v1 |
| https://arxiv.org/pdf/1906.01524v1.pdf | |
| PWC | https://paperswithcode.com/paper/text-based-editing-of-talking-head-video |
| Repo | |
| Framework | |
Visual Cue Integration for Small Target Motion Detection in Natural Cluttered Backgrounds
| Title | Visual Cue Integration for Small Target Motion Detection in Natural Cluttered Backgrounds |
| Authors | Hongxin Wang, Jigen Peng, Qinbing Fu, Huatian Wang, Shigang Yue |
| Abstract | The robust detection of small targets against cluttered background is important for future artificial visual systems in searching and tracking applications. The insects’ visual systems have demonstrated excellent ability to avoid predators, find prey or identify conspecifics - which always appear as small dim speckles in the visual field. Build a computational model of the insects’ visual pathways could provide effective solutions to detect small moving targets. Although a few visual system models have been proposed, they only make use of small-field visual features for motion detection and their detection results often contain a number of false positives. To address this issue, we develop a new visual system model for small target motion detection against cluttered moving backgrounds. Compared to the existing models, the small-field and wide-field visual features are separately extracted by two motion-sensitive neurons to detect small target motion and background motion. These two types of motion information are further integrated to filter out false positives. Extensive experiments showed that the proposed model can outperform the existing models in terms of detection rates. |
| Tasks | Motion Detection |
| Published | 2019-03-18 |
| URL | http://arxiv.org/abs/1903.07546v1 |
| http://arxiv.org/pdf/1903.07546v1.pdf | |
| PWC | https://paperswithcode.com/paper/visual-cue-integration-for-small-target |
| Repo | |
| Framework | |
Information Theoretic Lower Bounds on Negative Log Likelihood
| Title | Information Theoretic Lower Bounds on Negative Log Likelihood |
| Authors | Luis A. Lastras |
| Abstract | In this article we use rate-distortion theory, a branch of information theory devoted to the problem of lossy compression, to shed light on an important problem in latent variable modeling of data: is there room to improve the model? One way to address this question is to find an upper bound on the probability (equivalently a lower bound on the negative log likelihood) that the model can assign to some data as one varies the prior and/or the likelihood function in a latent variable model. The core of our contribution is to formally show that the problem of optimizing priors in latent variable models is exactly an instance of the variational optimization problem that information theorists solve when computing rate-distortion functions, and then to use this to derive a lower bound on negative log likelihood. Moreover, we will show that if changing the prior can improve the log likelihood, then there is a way to change the likelihood function instead and attain the same log likelihood, and thus rate-distortion theory is of relevance to both optimizing priors as well as optimizing likelihood functions. We will experimentally argue for the usefulness of quantities derived from rate-distortion theory in latent variable modeling by applying them to a problem in image modeling. |
| Tasks | Latent Variable Models |
| Published | 2019-04-12 |
| URL | http://arxiv.org/abs/1904.06395v1 |
| http://arxiv.org/pdf/1904.06395v1.pdf | |
| PWC | https://paperswithcode.com/paper/information-theoretic-lower-bounds-on-1 |
| Repo | |
| Framework | |
Quantifying Teaching Behaviour in Robot Learning from Demonstration
| Title | Quantifying Teaching Behaviour in Robot Learning from Demonstration |
| Authors | Aran Sena, Matthew J Howard |
| Abstract | Learning from demonstration allows for rapid deployment of robot manipulators to a great many tasks, by relying on a person showing the robot what to do rather than programming it. While this approach provides many opportunities, measuring, evaluating and improving the person’s teaching ability has remained largely unexplored in robot manipulation research. To this end, a model for learning from demonstration is presented here which incorporates the teacher’s understanding of, and influence on, the learner. The proposed model is used to clarify the teacher’s objectives during learning from demonstration, providing new views on how teaching failures and efficiency can be defined. The benefit of this approach is shown in two experiments (N=30 and N=36, respectively), which highlight the difficulty teachers have in providing effective demonstrations, and show how ~169-180% improvement in teaching efficiency can be achieved through evaluation and feedback shaped by the proposed framework, relative to unguided teaching. |
| Tasks | |
| Published | 2019-05-10 |
| URL | https://arxiv.org/abs/1905.04218v1 |
| https://arxiv.org/pdf/1905.04218v1.pdf | |
| PWC | https://paperswithcode.com/paper/quantifying-teaching-behaviour-in-robot |
| Repo | |
| Framework | |
ASAC: Active Sensing using Actor-Critic models
| Title | ASAC: Active Sensing using Actor-Critic models |
| Authors | Jinsung Yoon, James Jordon, Mihaela van der Schaar |
| Abstract | Deciding what and when to observe is critical when making observations is costly. In a medical setting where observations can be made sequentially, making these observations (or not) should be an active choice. We refer to this as the active sensing problem. In this paper, we propose a novel deep learning framework, which we call ASAC (Active Sensing using Actor-Critic models) to address this problem. ASAC consists of two networks: a selector network and a predictor network. The selector network uses previously selected observations to determine what should be observed in the future. The predictor network uses the observations selected by the selector network to predict a label, providing feedback to the selector network (well-selected variables should be predictive of the label). The goal of the selector network is then to select variables that balance the cost of observing the selected variables with their predictive power; we wish to preserve the conditional label distribution. During training, we use the actor-critic models to allow the loss of the selector to be “back-propagated” through the sampling process. The selector network “acts” by selecting future observations to make. The predictor network acts as a “critic” by feeding predictive errors for the selected variables back to the selector network. In our experiments, we show that ASAC significantly outperforms state-of-the-arts in two real-world medical datasets. |
| Tasks | |
| Published | 2019-06-16 |
| URL | https://arxiv.org/abs/1906.06796v1 |
| https://arxiv.org/pdf/1906.06796v1.pdf | |
| PWC | https://paperswithcode.com/paper/asac-active-sensing-using-actor-critic-models |
| Repo | |
| Framework | |
Shapes and Context: In-the-Wild Image Synthesis & Manipulation
| Title | Shapes and Context: In-the-Wild Image Synthesis & Manipulation |
| Authors | Aayush Bansal, Yaser Sheikh, Deva Ramanan |
| Abstract | We introduce a data-driven approach for interactively synthesizing in-the-wild images from semantic label maps. Our approach is dramatically different from recent work in this space, in that we make use of no learning. Instead, our approach uses simple but classic tools for matching scene context, shapes, and parts to a stored library of exemplars. Though simple, this approach has several notable advantages over recent work: (1) because nothing is learned, it is not limited to specific training data distributions (such as cityscapes, facades, or faces); (2) it can synthesize arbitrarily high-resolution images, limited only by the resolution of the exemplar library; (3) by appropriately composing shapes and parts, it can generate an exponentially large set of viable candidate output images (that can say, be interactively searched by a user). We present results on the diverse COCO dataset, significantly outperforming learning-based approaches on standard image synthesis metrics. Finally, we explore user-interaction and user-controllability, demonstrating that our system can be used as a platform for user-driven content creation. |
| Tasks | Image Generation |
| Published | 2019-06-11 |
| URL | https://arxiv.org/abs/1906.04728v1 |
| https://arxiv.org/pdf/1906.04728v1.pdf | |
| PWC | https://paperswithcode.com/paper/shapes-and-context-in-the-wild-image-1 |
| Repo | |
| Framework | |
