January 28, 2020

3067 words 15 mins read

Paper Group ANR 830

Unsupervised Feature Learning in Remote Sensing. Generic Ontology Design Patterns at Work. Do Autonomous Agents Benefit from Hearing?. Task Classification Model for Visual Fixation, Exploration, and Search. From Dark Matter to Galaxies with Convolutional Networks. Graph Representation Learning via Hard and Channel-Wise Attention Networks. A Simple …

Unsupervised Feature Learning in Remote Sensing


Title	Unsupervised Feature Learning in Remote Sensing
Authors	Aaron Reite, Scott Kangas, Zackery Steck, Steven Goley, Jonathan Von Stroh, Steven Forsyth
Abstract	The need for labeled data is among the most common and well-known practical obstacles to deploying deep learning algorithms to solve real-world problems. The current generation of learning algorithms requires a large volume of data labeled according to a static and pre-defined schema. Conversely, humans can quickly learn generalizations based on large quantities of unlabeled data, and turn these generalizations into classifications using spontaneous labels, often including labels not seen before. We apply a state-of-the-art unsupervised learning algorithm to the noisy and extremely imbalanced xView data set to train a feature extractor that adapts to several tasks: visual similarity search that performs well on both common and rare classes; identifying outliers within a labeled data set; and learning a natural class hierarchy automatically.
Tasks
Published	2019-08-07
URL	https://arxiv.org/abs/1908.02877v1
PDF	https://arxiv.org/pdf/1908.02877v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-feature-learning-in-remote
Repo
Framework

Generic Ontology Design Patterns at Work


Title	Generic Ontology Design Patterns at Work
Authors	Bernd Krieg-Brückner, Till Mossakowski, Fabian Neuhaus
Abstract	Generic Ontology Design Patterns, GODPs, are defined in Generic DOL, an extension of DOL, the Distributed Ontology, Model and Specification Language, and implemented using Heterogeneous Tool Set. Parameters such as classes, properties, individuals, or whole ontologies may be instantiated with arguments in a host ontology. The potential of Generic DOL is illustrated with GODPs for an example from the literature, namely the Role design pattern. We also discuss how larger GODPs may be composed by instantiating smaller GODPs.
Tasks
Published	2019-06-20
URL	https://arxiv.org/abs/1906.08724v1
PDF	https://arxiv.org/pdf/1906.08724v1.pdf
PWC	https://paperswithcode.com/paper/generic-ontology-design-patterns-at-work
Repo
Framework

Do Autonomous Agents Benefit from Hearing?


Title	Do Autonomous Agents Benefit from Hearing?
Authors	Abraham Woubie, Anssi Kanervisto, Janne Karttunen, Ville Hautamaki
Abstract	Mapping states to actions in deep reinforcement learning is mainly based on visual information. The commonly used approach for dealing with visual information is to extract pixels from images and use them as state representation for reinforcement learning agent. But, any vision only agent is handicapped by not being able to sense audible cues. Using hearing, animals are able to sense targets that are outside of their visual range. In this work, we propose the use of audio as complementary information to visual only in state representation. We assess the impact of such multi-modal setup in reach-the-goal tasks in ViZDoom environment. Results show that the agent improves its behavior when visual information is accompanied with audio features.
Tasks
Published	2019-05-10
URL	https://arxiv.org/abs/1905.04192v1
PDF	https://arxiv.org/pdf/1905.04192v1.pdf
PWC	https://paperswithcode.com/paper/do-autonomous-agents-benefit-from-hearing
Repo
Framework

Task Classification Model for Visual Fixation, Exploration, and Search


Title	Task Classification Model for Visual Fixation, Exploration, and Search
Authors	Ayush Kumar, Anjul Tyagi, Michael Burch, Daniel Weiskopf, Klaus Mueller
Abstract	Yarbus’ claim to decode the observer’s task from eye movements has received mixed reactions. In this paper, we have supported the hypothesis that it is possible to decode the task. We conducted an exploratory analysis on the dataset by projecting features and data points into a scatter plot to visualize the nuance properties for each task. Following this analysis, we eliminated highly correlated features before training an SVM and Ada Boosting classifier to predict the tasks from this filtered eye movements data. We achieve an accuracy of 95.4% on this task classification problem and hence, support the hypothesis that task classification is possible from a user’s eye movement data.
Tasks
Published	2019-07-29
URL	https://arxiv.org/abs/1907.12635v1
PDF	https://arxiv.org/pdf/1907.12635v1.pdf
PWC	https://paperswithcode.com/paper/task-classification-model-for-visual-fixation
Repo
Framework

From Dark Matter to Galaxies with Convolutional Networks


Title	From Dark Matter to Galaxies with Convolutional Networks
Authors	Xinyue Zhang, Yanfang Wang, Wei Zhang, Yueqiu Sun, Siyu He, Gabriella Contardo, Francisco Villaescusa-Navarro, Shirley Ho
Abstract	Cosmological surveys aim at answering fundamental questions about our Universe, including the nature of dark matter or the reason of unexpected accelerated expansion of the Universe. In order to answer these questions, two important ingredients are needed: 1) data from observations and 2) a theoretical model that allows fast comparison between observation and theory. Most of the cosmological surveys observe galaxies, which are very difficult to model theoretically due to the complicated physics involved in their formation and evolution; modeling realistic galaxies over cosmological volumes requires running computationally expensive hydrodynamic simulations that can cost millions of CPU hours. In this paper, we propose to use deep learning to establish a mapping between the 3D galaxy distribution in hydrodynamic simulations and its underlying dark matter distribution. One of the major challenges in this pursuit is the very high sparsity in the predicted galaxy distribution. To this end, we develop a two-phase convolutional neural network architecture to generate fast galaxy catalogues, and compare our results against a standard cosmological technique. We find that our proposed approach either outperforms or is competitive with traditional cosmological techniques. Compared to the common methods used in cosmology, our approach also provides a nice trade-off between time-consumption (comparable to fastest benchmark in the literature) and the quality and accuracy of the predicted simulation. In combination with current and upcoming data from cosmological observations, our method has the potential to answer fundamental questions about our Universe with the highest accuracy.
Tasks
Published	2019-02-15
URL	http://arxiv.org/abs/1902.05965v2
PDF	http://arxiv.org/pdf/1902.05965v2.pdf
PWC	https://paperswithcode.com/paper/from-dark-matter-to-galaxies-with
Repo
Framework

Graph Representation Learning via Hard and Channel-Wise Attention Networks


Title	Graph Representation Learning via Hard and Channel-Wise Attention Networks
Authors	Hongyang Gao, Shuiwang Ji
Abstract	Attention operators have been widely applied in various fields, including computer vision, natural language processing, and network embedding learning. Attention operators on graph data enables learnable weights when aggregating information from neighboring nodes. However, graph attention operators (GAOs) consume excessive computational resources, preventing their applications on large graphs. In addition, GAOs belong to the family of soft attention, instead of hard attention, which has been shown to yield better performance. In this work, we propose novel hard graph attention operator (hGAO) and channel-wise graph attention operator (cGAO). hGAO uses the hard attention mechanism by attending to only important nodes. Compared to GAO, hGAO improves performance and saves computational cost by only attending to important nodes. To further reduce the requirements on computational resources, we propose the cGAO that performs attention operations along channels. cGAO avoids the dependency on the adjacency matrix, leading to dramatic reductions in computational resource requirements. Experimental results demonstrate that our proposed deep models with the new operators achieve consistently better performance. Comparison results also indicates that hGAO achieves significantly better performance than GAO on both node and graph embedding tasks. Efficiency comparison shows that our cGAO leads to dramatic savings in computational resources, making them applicable to large graphs.
Tasks	Graph Classification, Graph Embedding, Graph Representation Learning, Network Embedding, Representation Learning
Published	2019-07-05
URL	https://arxiv.org/abs/1907.04652v1
PDF	https://arxiv.org/pdf/1907.04652v1.pdf
PWC	https://paperswithcode.com/paper/graph-representation-learning-via-hard-and
Repo
Framework

A Simple and Efficient Method to Compute a Single Linkage Dendrogram


Title	A Simple and Efficient Method to Compute a Single Linkage Dendrogram
Authors	Huanbiao Zhu, Werner Stuetzle
Abstract	We address the problem of computing a single linkage dendrogram. A possible approach is to: (i) Form an edge weighted graph $G$ over the data, with edge weights reflecting dissimilarities. (ii) Calculate the MST $T$ of $G$. (iii) Break the longest edge of $T$ thereby splitting it into subtrees $T_L$, $T_R$. (iv) Apply the splitting process recursively to the subtrees. This approach has the attractive feature that Prim’s algorithm for MST construction calculates distances as needed, and hence there is no need to ever store the inter-point distance matrix. The recursive partitioning algorithm requires us to determine the vertices (and edges) of $T_L$ and $T_R$. We show how this can be done easily and efficiently using information generated by Prim’s algorithm without any additional computational cost.
Tasks
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00223v1
PDF	https://arxiv.org/pdf/1911.00223v1.pdf
PWC	https://paperswithcode.com/paper/a-simple-and-efficient-method-to-compute-a
Repo
Framework

ModelHub.AI: Dissemination Platform for Deep Learning Models


Title	ModelHub.AI: Dissemination Platform for Deep Learning Models
Authors	Ahmed Hosny, Michael Schwier, Christoph Berger, Evin P Örnek, Mehmet Turan, Phi V Tran, Leon Weninger, Fabian Isensee, Klaus H Maier-Hein, Richard McKinley, Michael T Lu, Udo Hoffmann, Bjoern Menze, Spyridon Bakas, Andriy Fedorov, Hugo JWL Aerts
Abstract	Recent advances in artificial intelligence research have led to a profusion of studies that apply deep learning to problems in image analysis and natural language processing among others. Additionally, the availability of open-source computational frameworks has lowered the barriers to implementing state-of-the-art methods across multiple domains. Albeit leading to major performance breakthroughs in some tasks, effective dissemination of deep learning algorithms remains challenging, inhibiting reproducibility and benchmarking studies, impeding further validation, and ultimately hindering their effectiveness in the cumulative scientific progress. In developing a platform for sharing research outputs, we present ModelHub.AI (www.modelhub.ai), a community-driven container-based software engine and platform for the structured dissemination of deep learning models. For contributors, the engine controls data flow throughout the inference cycle, while the contributor-facing standard template exposes model-specific functions including inference, as well as pre- and post-processing. Python and RESTful Application programming interfaces (APIs) enable users to interact with models hosted on ModelHub.AI and allows both researchers and developers to utilize models out-of-the-box. ModelHub.AI is domain-, data-, and framework-agnostic, catering to different workflows and contributors’ preferences.
Tasks
Published	2019-11-26
URL	https://arxiv.org/abs/1911.13218v1
PDF	https://arxiv.org/pdf/1911.13218v1.pdf
PWC	https://paperswithcode.com/paper/modelhubai-dissemination-platform-for-deep
Repo
Framework

Variational Student: Learning Compact and Sparser Networks in Knowledge Distillation Framework


Title	Variational Student: Learning Compact and Sparser Networks in Knowledge Distillation Framework
Authors	Srinidhi Hegde, Ranjitha Prasad, Ramya Hebbalaguppe, Vishwajith Kumar
Abstract	The holy grail in deep neural network research is porting the memory- and computation-intensive network models on embedded platforms with a minimal compromise in model accuracy. To this end, we propose a novel approach, termed as Variational Student, where we reap the benefits of compressibility of the knowledge distillation (KD) framework, and sparsity inducing abilities of variational inference (VI) techniques. Essentially, we build a sparse student network, whose sparsity is induced by the variational parameters found via optimizing a loss function based on VI, leveraging the knowledge learnt by an accurate but complex pre-trained teacher network. Further, for sparsity enhancement, we also employ a Block Sparse Regularizer on a concatenated tensor of teacher and student network weights. We demonstrate that the marriage of KD and the VI techniques inherits compression properties from the KD framework, and enhances levels of sparsity from the VI approach, with minimal compromise in the model accuracy. We benchmark our results on LeNet MLP and VGGNet (CNN) and illustrate a memory footprint reduction of 64x and 213x on these MLP and CNN variants, respectively, without a need to retrain the teacher network. Furthermore, in the low data regime, we observed that our method outperforms state-of-the-art Bayesian techniques in terms of accuracy.
Tasks
Published	2019-10-26
URL	https://arxiv.org/abs/1910.12061v1
PDF	https://arxiv.org/pdf/1910.12061v1.pdf
PWC	https://paperswithcode.com/paper/variational-student-learning-compact-and
Repo
Framework

Robust Exploration with Tight Bayesian Plausibility Sets


Title	Robust Exploration with Tight Bayesian Plausibility Sets
Authors	Reazul H. Russel, Tianyi Gu, Marek Petrik
Abstract	Optimism about the poorly understood states and actions is the main driving force of exploration for many provably-efficient reinforcement learning algorithms. We propose optimism in the face of sensible value functions (OFVF)- a novel data-driven Bayesian algorithm to constructing Plausibility sets for MDPs to explore robustly minimizing the worst case exploration cost. The method computes policies with tighter optimistic estimates for exploration by introducing two new ideas. First, it is based on Bayesian posterior distributions rather than distribution-free bounds. Second, OFVF does not construct plausibility sets as simple confidence intervals. Confidence intervals as plausibility sets are a sufficient but not a necessary condition. OFVF uses the structure of the value function to optimize the location and shape of the plausibility set to guarantee upper bounds directly without necessarily enforcing the requirement for the set to be a confidence interval. OFVF proceeds in an episodic manner, where the duration of the episode is fixed and known. Our algorithm is inherently Bayesian and can leverage prior information. Our theoretical analysis shows the robustness of OFVF, and the empirical results demonstrate its practical promise.
Tasks
Published	2019-04-17
URL	http://arxiv.org/abs/1904.08528v1
PDF	http://arxiv.org/pdf/1904.08528v1.pdf
PWC	https://paperswithcode.com/paper/robust-exploration-with-tight-bayesian
Repo
Framework

Deep inspection: an electrical distribution pole parts study via deep neural networks


Title	Deep inspection: an electrical distribution pole parts study via deep neural networks
Authors	Liangchen Liu, Teng Zhang, Kun Zhao, Arnold Wiliem, Kieren Astin-Walmsley, Brian Lovell
Abstract	Electrical distribution poles are important assets in electricity supply. These poles need to be maintained in good condition to ensure they protect community safety, maintain reliability of supply, and meet legislative obligations. However, maintaining such a large volumes of assets is an expensive and challenging task. To address this, recent approaches utilise imagery data captured from helicopter and/or drone inspections. Whilst reducing the cost for manual inspection, manual analysis on each image is still required. As such, several image-based automated inspection systems have been proposed. In this paper, we target two major challenges: tiny object detection and extremely imbalanced datasets, which currently hinder the wide deployment of the automatic inspection. We propose a novel two-stage zoom-in detection method to gradually focus on the object of interest. To address the imbalanced dataset problem, we propose the resampling as well as reweighting schemes to iteratively adapt the model to the large intra-class variation of major class and balance the contributions to the loss from each class. Finally, we integrate these components together and devise a novel automatic inspection framework. Extensive experiments demonstrate that our proposed approaches are effective and can boost the performance compared to the baseline methods.
Tasks	Object Detection
Published	2019-07-16
URL	https://arxiv.org/abs/1907.06844v1
PDF	https://arxiv.org/pdf/1907.06844v1.pdf
PWC	https://paperswithcode.com/paper/deep-inspection-an-electrical-distribution
Repo
Framework

Differential Dynamic Programming for Multi-Phase Rigid Contact Dynamics


Title	Differential Dynamic Programming for Multi-Phase Rigid Contact Dynamics
Authors	Rohan Budhiraja, Justin Carpentier, Carlos Mastalli, Nicolas Mansard
Abstract	A common strategy today to generate efficient locomotion movements is to split the problem into two consecutive steps: the first one generates the contact sequence together with the centroidal trajectory, while the second one computes the whole-body trajectory that follows the centroidal pattern. Yet the second step is generally handled by a simple program such as an inverse kinematics solver. In contrast, we propose to compute the whole-body trajectory by using a local optimal control solver, namely Differential Dynamic Programming (DDP). Our method produces more efficient motions, with lower forces and smaller impacts, by exploiting the Angular Momentum (AM). With this aim, we propose an original DDP formulation exploiting the Karush-Kuhn-Tucker constraint of the rigid contact model. We experimentally show the importance of this approach by executing large steps walking on the real HRP-2 robot, and by solving the problem of attitude control under the absence of external forces.
Tasks
Published	2019-04-10
URL	http://arxiv.org/abs/1904.05072v1
PDF	http://arxiv.org/pdf/1904.05072v1.pdf
PWC	https://paperswithcode.com/paper/differential-dynamic-programming-for-multi
Repo
Framework

Why Learning of Large-Scale Neural Networks Behaves Like Convex Optimization


Title	Why Learning of Large-Scale Neural Networks Behaves Like Convex Optimization
Authors	Hui Jiang
Abstract	In this paper, we present some theoretical work to explain why simple gradient descent methods are so successful in solving non-convex optimization problems in learning large-scale neural networks (NN). After introducing a mathematical tool called canonical space, we have proved that the objective functions in learning NNs are convex in the canonical model space. We further elucidate that the gradients between the original NN model space and the canonical space are related by a pointwise linear transformation, which is represented by the so-called disparity matrix. Furthermore, we have proved that gradient descent methods surely converge to a global minimum of zero loss provided that the disparity matrices maintain full rank. If this full-rank condition holds, the learning of NNs behaves in the same way as normal convex optimization. At last, we have shown that the chance to have singular disparity matrices is extremely slim in large NNs. In particular, when over-parameterized NNs are randomly initialized, the gradient decent algorithms converge to a global minimum of zero loss in probability.
Tasks
Published	2019-03-06
URL	http://arxiv.org/abs/1903.02140v1
PDF	http://arxiv.org/pdf/1903.02140v1.pdf
PWC	https://paperswithcode.com/paper/why-learning-of-large-scale-neural-networks
Repo
Framework

AdaBoost-assisted Extreme Learning Machine for Efficient Online Sequential Classification


Title	AdaBoost-assisted Extreme Learning Machine for Efficient Online Sequential Classification
Authors	Yi-Ta Chen, Yu-Chuan Chuang, An-Yeu, Wu
Abstract	In this paper, we propose an AdaBoost-assisted extreme learning machine for efficient online sequential classification (AOS-ELM). In order to achieve better accuracy in online sequential learning scenarios, we utilize the cost-sensitive algorithm-AdaBoost, which diversifying the weak classifiers, and adding the forgetting mechanism, which stabilizing the performance during the training procedure. Hence, AOS-ELM adapts better to sequentially arrived data compared with other voting based methods. The experiment results show AOS-ELM can achieve 94.41% accuracy on MNIST dataset, which is the theoretical accuracy bound performed by an original batch learning algorithm, AdaBoost-ELM. Moreover, with the forgetting mechanism, the standard deviation of accuracy during the online sequential learning process is reduced to 8.26x.
Tasks
Published	2019-09-16
URL	https://arxiv.org/abs/1909.07115v1
PDF	https://arxiv.org/pdf/1909.07115v1.pdf
PWC	https://paperswithcode.com/paper/adaboost-assisted-extreme-learning-machine
Repo
Framework

Online influence, offline violence: Linguistic responses to the ‘Unite the Right’ rally


Title	Online influence, offline violence: Linguistic responses to the ‘Unite the Right’ rally
Authors	Isabelle van der Vegt, Maximilian Mozes, Paul Gill, Bennett Kleinberg
Abstract	The media frequently describes the 2017 Charlottesville ‘Unite the Right’ rally as a turning point for the alt-right and white supremacist movements. Related research into social movements also suggests that the media attention and public discourse concerning the rally may have influenced the alt-right. Empirical evidence for these claims is largely lacking. The current study investigates potential effects of the rally by examining a dataset of 7,142 YouTube video transcripts from alt-right and progressive channels. We examine sentiment surrounding the ten most frequent keywords (single words and word pairs) in transcripts from each group, eight weeks before to eight weeks after the rally. In the majority of cases, no significant differences in sentiment were found within and between the alt-right and progressive groups, both pre- and post-Charlottesville. However, we did observe more negative sentiment trends surrounding ‘Bernie Sanders’ and ‘black people’ in the alt-right and progressive groups, respectively. We also observed more negative sentiment after the rally regarding ‘Democratic Party’ in the alt-right videos compared to the progressive videos. We suggest that the observed results potentially reflect minor changes in political sentiment before and after the rally, as well as differences in political sentiment between the alt-right and progressive groups in general.
Tasks
Published	2019-08-30
URL	https://arxiv.org/abs/1908.11599v1
PDF	https://arxiv.org/pdf/1908.11599v1.pdf
PWC	https://paperswithcode.com/paper/online-influence-offline-violence-linguistic
Repo
Framework