Paper Group ANR 219
TeaNet: universal neural network interatomic potential inspired by iterative electronic relaxations. Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations. Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails. Manifold Optimisation Assisted Gaussian Variational Approximation. A novel approach to mod …
TeaNet: universal neural network interatomic potential inspired by iterative electronic relaxations
| Title | TeaNet: universal neural network interatomic potential inspired by iterative electronic relaxations |
| Authors | So Takamoto, Satoshi Izumi, Ju Li |
| Abstract | A universal interatomic potential applicable to arbitrary elements and structures is urgently needed in computational materials science. Graph convolution-based neural network is a promising approach by virtue of its ability to express complex relations. Thus far, it has been thought to represent a completely different approach from physics-based interatomic potentials. In this paper, we show that these two methods can be regarded as different representations of the same tight-binding electronic relaxation framework, where atom-based and overlap integral or “bond”-based Hamiltonian information are propagated in a directional fashion. Based on this unified view, we propose a new model, named the tensor embedded atom network (TeaNet), where the stacked network model is associated with the electronic total energy relaxation calculation. Furthermore, Tersoff-style angular interaction is translated into graph convolution architecture through the incorporation of Euclidean tensor values. Our model can represent and transfer spatial information. TeaNet shows great performance in both the robustness of interatomic potentials and the expressive power of neural networks. We demonstrate that arbitrary chemistry involving the first 18 elements on the periodic table (H to Ar) can be realized by our model, including C-H molecular structures, metals, amorphous SiO${}_2$, and water. |
| Tasks | |
| Published | 2019-12-02 |
| URL | https://arxiv.org/abs/1912.01398v1 |
| https://arxiv.org/pdf/1912.01398v1.pdf | |
| PWC | https://paperswithcode.com/paper/teanet-universal-neural-network-interatomic |
| Repo | |
| Framework | |
Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations
| Title | Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations |
| Authors | Shuran Song, Andy Zeng, Johnny Lee, Thomas Funkhouser |
| Abstract | Intelligent manipulation benefits from the capacity to flexibly control an end-effector with high degrees of freedom (DoF) and dynamically react to the environment. However, due to the challenges of collecting effective training data and learning efficiently, most grasping algorithms today are limited to top-down movements and open-loop execution. In this work, we propose a new low-cost hardware interface for collecting grasping demonstrations by people in diverse environments. Leveraging this data, we show that it is possible to train a robust end-to-end 6DoF closed-loop grasping model with reinforcement learning that transfers to real robots. A key aspect of our grasping model is that it uses ``action-view’’ based rendering to simulate future states with respect to different possible actions. By evaluating these states using a learned value function (Q-function), our method is able to better select corresponding actions that maximize total rewards (i.e., grasping success). Our final grasping system is able to achieve reliable 6DoF closed-loop grasping of novel objects across various scene configurations, as well as dynamic scenes with moving objects. | |
| Tasks | |
| Published | 2019-12-09 |
| URL | https://arxiv.org/abs/1912.04344v1 |
| https://arxiv.org/pdf/1912.04344v1.pdf | |
| PWC | https://paperswithcode.com/paper/grasping-in-the-wildlearning-6dof-closed-loop |
| Repo | |
| Framework | |
Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails
| Title | Virtual-to-Real-World Transfer Learning for Robots on Wilderness Trails |
| Authors | Michael L. Iuzzolino, Michael E. Walker, Daniel Szafir |
| Abstract | Robots hold promise in many scenarios involving outdoor use, such as search-and-rescue, wildlife management, and collecting data to improve environment, climate, and weather forecasting. However, autonomous navigation of outdoor trails remains a challenging problem. Recent work has sought to address this issue using deep learning. Although this approach has achieved state-of-the-art results, the deep learning paradigm may be limited due to a reliance on large amounts of annotated training data. Collecting and curating training datasets may not be feasible or practical in many situations, especially as trail conditions may change due to seasonal weather variations, storms, and natural erosion. In this paper, we explore an approach to address this issue through virtual-to-real-world transfer learning using a variety of deep learning models trained to classify the direction of a trail in an image. Our approach utilizes synthetic data gathered from virtual environments for model training, bypassing the need to collect a large amount of real images of the outdoors. We validate our approach in three main ways. First, we demonstrate that our models achieve classification accuracies upwards of 95% on our synthetic data set. Next, we utilize our classification models in the control system of a simulated robot to demonstrate feasibility. Finally, we evaluate our models on real-world trail data and demonstrate the potential of virtual-to-real-world transfer learning. |
| Tasks | Autonomous Navigation, Transfer Learning, Weather Forecasting |
| Published | 2019-01-17 |
| URL | http://arxiv.org/abs/1901.05599v1 |
| http://arxiv.org/pdf/1901.05599v1.pdf | |
| PWC | https://paperswithcode.com/paper/virtual-to-real-world-transfer-learning-for |
| Repo | |
| Framework | |
Manifold Optimisation Assisted Gaussian Variational Approximation
| Title | Manifold Optimisation Assisted Gaussian Variational Approximation |
| Authors | Bingxin Zhou, Junbin Gao, Minh-Ngoc Tran, Richard Gerlach |
| Abstract | Variational approximation methods are a way to approximate the posterior in Bayesian inference especially when the dataset has a large volume or high dimension. Factor covariance structure was introduced in previous work with three restrictions to handle the problem of computational infeasibility in Gaussian approximation. However, the three strong constraints on the covariance matrix could possibly break down during the process of the structure optimization, and the identification issue could still possibly exist within the final approximation. In this paper, we consider two types of manifold parameterization, Stiefel manifold and Grassmann manifold, to address the problems. Moreover, the Riemannian stochastic gradient descent method is applied to solve the resulting optimization problem while maintaining the orthogonal factors. Results from two experiments demonstrate that our model fixes the potential issue of the previous method with comparable accuracy and competitive converge speed even in high-dimensional problems. |
| Tasks | Bayesian Inference |
| Published | 2019-02-11 |
| URL | http://arxiv.org/abs/1902.03718v1 |
| http://arxiv.org/pdf/1902.03718v1.pdf | |
| PWC | https://paperswithcode.com/paper/manifold-optimisation-assisted-gaussian |
| Repo | |
| Framework | |
A novel approach to model exploration for value function learning
| Title | A novel approach to model exploration for value function learning |
| Authors | Zlatan Ajanovic, Halil Beglerovic, Bakir Lacevic |
| Abstract | Planning and Learning are complementary approaches. Planning relies on deliberative reasoning about the current state and sequence of future reachable states to solve the problem. Learning, on the other hand, is focused on improving system performance based on experience or available data. Learning to improve the performance of planning based on experience in similar, previously solved problems, is ongoing research. One approach is to learn Value function (cost-to-go) which can be used as heuristics for speeding up search-based planning. Existing approaches in this direction use the results of the previous search for learning the heuristics. In this work, we present a search-inspired approach of systematic model exploration for the learning of the value function which does not stop when a plan is available but rather prolongs search such that not only resulting optimal path is used but also extended region around the optimal path. This, in turn, improves both the efficiency and robustness of successive planning. Additionally, the effect of losing admissibility by using ML heuristic is managed by bounding ML with other admissible heuristics. |
| Tasks | |
| Published | 2019-06-06 |
| URL | https://arxiv.org/abs/1906.02789v2 |
| https://arxiv.org/pdf/1906.02789v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-novel-approach-to-model-exploration-for |
| Repo | |
| Framework | |
The Cost of Privacy: Optimal Rates of Convergence for Parameter Estimation with Differential Privacy
| Title | The Cost of Privacy: Optimal Rates of Convergence for Parameter Estimation with Differential Privacy |
| Authors | T. Tony Cai, Yichen Wang, Linjun Zhang |
| Abstract | Privacy-preserving data analysis is a rising challenge in contemporary statistics, as the privacy guarantees of statistical methods are often achieved at the expense of accuracy. In this paper, we investigate the tradeoff between statistical accuracy and privacy in mean estimation and linear regression, under both the classical low-dimensional and modern high-dimensional settings. A primary focus is to establish minimax optimality for statistical estimation with the $(\varepsilon,\delta)$-differential privacy constraint. To this end, we find that classical lower bound arguments fail to yield sharp results, and new technical tools are called for. By refining the “tracing adversary” technique for lower bounds in the theoretical computer science literature, we formulate a general lower bound argument for minimax risks with differential privacy constraints, and apply this argument to high-dimensional mean estimation and linear regression problems. We also design computationally efficient algorithms that attain the minimax lower bounds up to a logarithmic factor. In particular, for the high-dimensional linear regression, a novel private iterative hard thresholding pursuit algorithm is proposed, based on a privately truncated version of stochastic gradient descent. The numerical performance of these algorithms is demonstrated by simulation studies and applications to real data containing sensitive information, for which privacy-preserving statistical methods are necessary. |
| Tasks | |
| Published | 2019-02-12 |
| URL | https://arxiv.org/abs/1902.04495v3 |
| https://arxiv.org/pdf/1902.04495v3.pdf | |
| PWC | https://paperswithcode.com/paper/the-cost-of-privacy-optimal-rates-of |
| Repo | |
| Framework | |
An Analysis of an Integrated Mathematical Modeling – Artificial Neural Network Approach for the Problems with a Limited Learning Dataset
| Title | An Analysis of an Integrated Mathematical Modeling – Artificial Neural Network Approach for the Problems with a Limited Learning Dataset |
| Authors | Szymon Buchaniec, Marek Gnatowski, Grzegorz Brus |
| Abstract | One of the most common and universal problems in science is to investigate a function. The prediction can be made by an Artificial Neural Network (ANN) or a mathematical model. Both approaches have their advantages and disadvantages. Mathematical models were sought as more trustworthy as their prediction is based on the laws of physics expressed in the form of mathematical equations. However, the majority of existing mathematical models include different empirical parameters, and both approaches inherit inevitable experimental errors. At the same time, the approximation of neural networks can reproduce the solution extremely well if fed with a sufficient amount of data. The difference is that an ANN requires big data to build its accurate approximation whereas a typical mathematical model needs just several data points to estimate an empirical constant. Therefore, the common problem that developer meet is the inaccuracy of mathematical models and artificial neural network. An another common challenge is the computational complexity of the mathematical models, or lack of data for a sufficient precision of the Artificial Neural Networks. In the presented paper those problems are addressed using the integration of a mathematical model with an artificial neural network. In the presented analysis, an ANN predicts just a part of the mathematical model and its weights and biases are adjusted based on the output of the mathematical model. The performance of Integrated Mathematical modeling - Artificial Neural Network (IMANN) is compared to a Dense Neural Network (DNN) with the use of the benchmarking functions. The obtained calculation results indicate that such an approach could lead to an increase of precision as well as limiting the data-set required for learning. |
| Tasks | |
| Published | 2019-11-08 |
| URL | https://arxiv.org/abs/1911.03404v2 |
| https://arxiv.org/pdf/1911.03404v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-new-interactive-mathematical-modeling |
| Repo | |
| Framework | |
Interpretable Few-Shot Learning via Linear Distillation
| Title | Interpretable Few-Shot Learning via Linear Distillation |
| Authors | Arip Asadulaev, Igor Kuznetsov, Andrey Filchenkov |
| Abstract | It is important to develop mathematically tractable models than can interpret knowledge extracted from the data and provide reasonable predictions. In this paper, we present a Linear Distillation Learning, a simple remedy to improve the performance of linear neural networks. Our approach is based on using a linear function for each class in a dataset, which is trained to simulate the output of a teacher linear network for each class separately. We tested our model on MNIST and Omniglot datasets in the Few-Shot learning manner. It showed better results than other interpretable models such as classical Logistic Regression. |
| Tasks | Few-Shot Learning, Omniglot |
| Published | 2019-06-13 |
| URL | https://arxiv.org/abs/1906.05431v2 |
| https://arxiv.org/pdf/1906.05431v2.pdf | |
| PWC | https://paperswithcode.com/paper/linear-distillation-learning |
| Repo | |
| Framework | |
A Scheme for Dynamic Risk-Sensitive Sequential Decision Making
| Title | A Scheme for Dynamic Risk-Sensitive Sequential Decision Making |
| Authors | Shuai Ma, Jia Yuan Yu, Ahmet Satir |
| Abstract | We present a scheme for sequential decision making with a risk-sensitive objective and constraints in a dynamic environment. A neural network is trained as an approximator of the mapping from parameter space to space of risk and policy with risk-sensitive constraints. For a given risk-sensitive problem, in which the objective and constraints are, or can be estimated by, functions of the mean and variance of return, we generate a synthetic dataset as training data. Parameters defining a targeted process might be dynamic, i.e., they might vary over time, so we sample them within specified intervals to deal with these dynamics. We show that: i). Most risk measures can be estimated using return variance; ii). By virtue of the state-augmentation transformation, practical problems modeled by Markov decision processes with stochastic rewards can be solved in a risk-sensitive scenario; and iii). The proposed scheme is validated by a numerical experiment. |
| Tasks | Decision Making |
| Published | 2019-07-09 |
| URL | https://arxiv.org/abs/1907.04269v1 |
| https://arxiv.org/pdf/1907.04269v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-scheme-for-dynamic-risk-sensitive |
| Repo | |
| Framework | |
Graph based Neural Networks for Event Factuality Prediction using Syntactic and Semantic Structures
| Title | Graph based Neural Networks for Event Factuality Prediction using Syntactic and Semantic Structures |
| Authors | Amir Pouran Ben Veyseh, Thien Huu Nguyen, Dejing Dou |
| Abstract | Event factuality prediction (EFP) is the task of assessing the degree to which an event mentioned in a sentence has happened. For this task, both syntactic and semantic information are crucial to identify the important context words. The previous work for EFP has only combined these information in a simple way that cannot fully exploit their coordination. In this work, we introduce a novel graph-based neural network for EFP that can integrate the semantic and syntactic information more effectively. Our experiments demonstrate the advantage of the proposed model for EFP. |
| Tasks | |
| Published | 2019-07-07 |
| URL | https://arxiv.org/abs/1907.03227v1 |
| https://arxiv.org/pdf/1907.03227v1.pdf | |
| PWC | https://paperswithcode.com/paper/graph-based-neural-networks-for-event |
| Repo | |
| Framework | |
Automatic Detection of ECG Abnormalities by using an Ensemble of Deep Residual Networks with Attention
| Title | Automatic Detection of ECG Abnormalities by using an Ensemble of Deep Residual Networks with Attention |
| Authors | Yang Liu, Runnan He, Kuanquan Wang, Qince Li, Qiang Sun, Na Zhao, Henggui Zhang |
| Abstract | Heart disease is one of the most common diseases causing morbidity and mortality. Electrocardiogram (ECG) has been widely used for diagnosing heart diseases for its simplicity and non-invasive property. Automatic ECG analyzing technologies are expected to reduce human working load and increase diagnostic efficacy. However, there are still some challenges to be addressed for achieving this goal. In this study, we develop an algorithm to identify multiple abnormalities from 12-lead ECG recordings. In the algorithm pipeline, several preprocessing methods are firstly applied on the ECG data for denoising, augmentation and balancing recording numbers of variant classes. In consideration of efficiency and consistency of data length, the recordings are padded or truncated into a medium length, where the padding/truncating time windows are selected randomly to sup-press overfitting. Then, the ECGs are used to train deep neural network (DNN) models with a novel structure that combines a deep residual network with an attention mechanism. Finally, an ensemble model is built based on these trained models to make predictions on the test data set. Our method is evaluated based on the test set of the First China ECG Intelligent Competition dataset by using the F1 metric that is regarded as the harmonic mean between the precision and recall. The resultant overall F1 score of the algorithm is 0.875, showing a promising performance and potential for practical use. |
| Tasks | Denoising |
| Published | 2019-08-27 |
| URL | https://arxiv.org/abs/1908.10088v1 |
| https://arxiv.org/pdf/1908.10088v1.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-detection-of-ecg-abnormalities-by |
| Repo | |
| Framework | |
An Interaction Framework for Studying Co-Creative AI
| Title | An Interaction Framework for Studying Co-Creative AI |
| Authors | Matthew Guzdial, Mark Riedl |
| Abstract | Machine learning has been applied to a number of creative, design-oriented tasks. However, it remains unclear how to best empower human users with these machine learning approaches, particularly those users without technical expertise. In this paper we propose a general framework for turn-based interaction between human users and AI agents designed to support human creativity, called {co-creative systems}. The framework can be used to better understand the space of possible designs of co-creative systems and reveal future research directions. We demonstrate how to apply this framework in conjunction with a pair of recent human subject studies, comparing between the four human-AI systems employed in these studies and generating hypotheses towards future studies. |
| Tasks | |
| Published | 2019-03-22 |
| URL | http://arxiv.org/abs/1903.09709v1 |
| http://arxiv.org/pdf/1903.09709v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-interaction-framework-for-studying-co |
| Repo | |
| Framework | |
Automated Fashion Size Normalization
| Title | Automated Fashion Size Normalization |
| Authors | Eddie S. J. Du, Chang Liu, David H. Wayne |
| Abstract | The ability to accurately predict the fit of fashion items and recommend the correct size is key to reducing merchandise returns in e-commerce. A critical prerequisite of fit prediction is size normalization, the mapping of product sizes across brands to a common space in which sizes can be compared. At present, size normalization is usually a time-consuming manual process. We propose a method to automate size normalization through the use of salesdata. The size mappings generated from our automated approaches are comparable to human-generated mappings. |
| Tasks | |
| Published | 2019-08-27 |
| URL | https://arxiv.org/abs/1908.09980v1 |
| https://arxiv.org/pdf/1908.09980v1.pdf | |
| PWC | https://paperswithcode.com/paper/automated-fashion-size-normalization |
| Repo | |
| Framework | |
CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors
| Title | CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors |
| Authors | Sanghamitra Dutta, Ziqian Bai, Tze Meng Low, Pulkit Grover |
| Abstract | This work proposes the first strategy to make distributed training of neural networks resilient to computing errors, a problem that has remained unsolved despite being first posed in 1956 by von Neumann. He also speculated that the efficiency and reliability of the human brain is obtained by allowing for low power but error-prone components with redundancy for error-resilience. It is surprising that this problem remains open, even as massive artificial neural networks are being trained on increasingly low-cost and unreliable processing units. Our coding-theory-inspired strategy, “CodeNet,” solves this problem by addressing three challenges in the science of reliable computing: (i) Providing the first strategy for error-resilient neural network training by encoding each layer separately; (ii) Keeping the overheads of coding (encoding/error-detection/decoding) low by obviating the need to re-encode the updated parameter matrices after each iteration from scratch. (iii) Providing a completely decentralized implementation with no central node (which is a single point of failure), allowing all primary computational steps to be error-prone. We theoretically demonstrate that CodeNet has higher error tolerance than replication, which we leverage to speed up computation time. Simultaneously, CodeNet requires lower redundancy than replication, and equal computational and communication costs in scaling sense. We first demonstrate the benefits of CodeNet in reducing expected computation time over replication when accounting for checkpointing. Our experiments show that CodeNet achieves the best accuracy-runtime tradeoff compared to both replication and uncoded strategies. CodeNet is a significant step towards biologically plausible neural network training, that could hold the key to orders of magnitude efficiency improvements. |
| Tasks | |
| Published | 2019-03-04 |
| URL | http://arxiv.org/abs/1903.01042v1 |
| http://arxiv.org/pdf/1903.01042v1.pdf | |
| PWC | https://paperswithcode.com/paper/codenet-training-large-scale-neural-networks |
| Repo | |
| Framework | |
Efficient Neural Task Adaptation by Maximum Entropy Initialization
| Title | Efficient Neural Task Adaptation by Maximum Entropy Initialization |
| Authors | Farshid Varno, Behrouz Haji Soleimani, Marzie Saghayi, Lisa Di Jorio, Stan Matwin |
| Abstract | Transferring knowledge from one neural network to another has been shown to be helpful for learning tasks with few training examples. Prevailing fine-tuning methods could potentially contaminate pre-trained features by comparably high energy random noise. This noise is mainly delivered from a careless replacement of task-specific parameters. We analyze theoretically such knowledge contamination for classification tasks and propose a practical and easy to apply method to trap and minimize the contaminant. In our approach, the entropy of the output estimates gets maximized initially and the first back-propagated error is stalled at the output of the last layer. Our proposed method not only outperforms the traditional fine-tuning, but also significantly speeds up the convergence of the learner. It is robust to randomness and independent of the choice of architecture. Overall, our experiments show that the power of transfer learning has been substantially underestimated so far. |
| Tasks | Transfer Learning |
| Published | 2019-05-25 |
| URL | https://arxiv.org/abs/1905.10698v2 |
| https://arxiv.org/pdf/1905.10698v2.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-neural-task-adaptation-by-maximum |
| Repo | |
| Framework | |
