April 3, 2020

# Paper Group AWR 74

Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud. Statistical power for cluster analysis. Multi-variate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows. Synonymous Generalization in Sequence-to-Sequence Recurrent Networks. CRYSPNet: Crystal Structure Predictions via Neural Network. NYTWIT: A Dataset o …

#### Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud

Title Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
Authors Weijing Shi, Ragunathan, Rajkumar
Abstract In this paper, we propose a graph neural network to detect objects from a LiDAR point cloud. Towards this end, we encode the point cloud efficiently in a fixed radius near-neighbors graph. We design a graph neural network, named Point-GNN, to predict the category and shape of the object that each vertex in the graph belongs to. In Point-GNN, we propose an auto-registration mechanism to reduce translation variance, and also design a box merging and scoring operation to combine detections from multiple vertices accurately. Our experiments on the KITTI benchmark show the proposed approach achieves leading accuracy using the point cloud alone and can even surpass fusion-based algorithms. Our results demonstrate the potential of using the graph neural network as a new approach for 3D object detection. The code is available https://github.com/WeijingShi/Point-GNN.
Tasks 3D Object Detection, Object Detection
Published 2020-03-02
URL https://arxiv.org/abs/2003.01251v1
PDF https://arxiv.org/pdf/2003.01251v1.pdf
PWC https://paperswithcode.com/paper/point-gnn-graph-neural-network-for-3d-object
Repo https://github.com/WeijingShi/Point-GNN
Framework tf

#### Statistical power for cluster analysis

Title Statistical power for cluster analysis
Authors E. S. Dalmaijer, C. L. Nord, D. E. Astle
Abstract Cluster algorithms are gaining in popularity due to their compelling ability to identify discrete subgroups in data, and their increasing accessibility in mainstream programming languages and statistical software. While researchers can follow guidelines to choose the right algorithms, and to determine what constitutes convincing clustering, there are no firmly established ways of computing a priori statistical power for cluster analysis. Here, we take a simulation approach to estimate power and classification accuracy for popular analysis pipelines. We systematically varied cluster size, number of clusters, number of different features between clusters, effect size within each different feature, and cluster covariance structure in generated datasets. We then subjected these datasets to common dimensionality reduction approaches (none, multi-dimensional scaling, or uniform manifold approximation and projection) and cluster algorithms (k-means, hierarchical agglomerative clustering with Ward linkage and Euclidean distance, or average linkage and cosine distance, HDBSCAN). Furthermore, we simulated additional datasets to explore the effect of sample size and cluster separation on statistical power and classification accuracy. We found that clustering outcomes were driven by large effect sizes or the accumulation of many smaller effects across features, and were mostly unaffected by differences in covariance structure. Sufficient statistical power can be achieved with relatively small samples (N=20 per subgroup), provided cluster separation is large ({\Delta}=4). Finally, we discuss whether fuzzy clustering (c-means) could provide a more parsimonious alternative for identifying separable multivariate normal distributions, particularly those with lower centroid separation.
Published 2020-03-01
URL https://arxiv.org/abs/2003.00381v1
PDF https://arxiv.org/pdf/2003.00381v1.pdf
PWC https://paperswithcode.com/paper/statistical-power-for-cluster-analysis
Repo https://github.com/esdalmaijer/cluster_power
Framework none

#### Multi-variate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows

Title Multi-variate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows
Authors Kashif Rasul, Abdul-Saboor Sheikh, Ingmar Schuster, Urs Bergmann, Roland Vollgraf
Abstract Time series forecasting is often fundamental to scientific and engineering problems and enables decision making. With ever increasing data set sizes, a trivial solution to scale up predictions is to assume independence between interacting time series. However, modeling statistical dependencies can improve accuracy and enable analysis of interaction effects. Deep learning methods are well suited for this problem, but multi-variate models often assume a simple parametric distribution and do not scale to high dimensions. In this work we model the multi-variate temporal dynamics of time series via an autoregressive deep learning model, where the data distribution is represented by a conditioned normalizing flow. This combination retains the power of autoregressive models, such as good performance in extrapolation into the future, with the flexibility of flows as a general purpose high-dimensional distribution model, while remaining computationally tractable. We show that it improves over the state-of-the-art for standard metrics on many real-world data sets with several thousand interacting time-series.
Tasks Decision Making, Time Series, Time Series Forecasting
Published 2020-02-14
URL https://arxiv.org/abs/2002.06103v1
PDF https://arxiv.org/pdf/2002.06103v1.pdf
PWC https://paperswithcode.com/paper/multi-variate-probabilistic-time-series
Repo https://github.com/zalandoresearch/pytorch-ts
Framework pytorch

#### Synonymous Generalization in Sequence-to-Sequence Recurrent Networks

Title Synonymous Generalization in Sequence-to-Sequence Recurrent Networks
Authors Ning Shi
Abstract When learning a language, people can quickly expand their understanding of the unknown content by using compositional skills, such as from two words “go” and “fast” to a new phrase “go fast.” In recent work of Lake and Baroni (2017), modern Sequence-to-Sequence(se12seq) Recurrent Neural Networks (RNNs) can make powerful zero-shot generalizations in specifically controlled experiments. However, there is a missing regarding the property of such strong generalization and its precise requirements. This paper explores this positive result in detail and defines this pattern as the synonymous generalization, an ability to recognize an unknown sequence by decomposing the difference between it and a known sequence as corresponding existing synonyms. To better investigate it, I introduce a new environment called Colorful Extended Cleanup World (CECW), which consists of complex commands paired with logical expressions. While demonstrating that sequential RNNs can perform synonymous generalizations on foreign commands, I conclude their prerequisites for success. I also propose a data augmentation method, which is successfully verified on the Geoquery (GEO) dataset, as a novel application of synonymous generalization for real cases.
Published 2020-03-14
URL https://arxiv.org/abs/2003.06658v1
PDF https://arxiv.org/pdf/2003.06658v1.pdf
PWC https://paperswithcode.com/paper/synonymous-generalization-in-sequence-to
Repo https://github.com/MrShininnnnn/CECW
Framework none

#### CRYSPNet: Crystal Structure Predictions via Neural Network

Title CRYSPNet: Crystal Structure Predictions via Neural Network
Authors Haotong Liang, Valentin Stanev, A. Gilad Kusne, Ichiro Takeuchi
Abstract Structure is the most basic and important property of crystalline solids; it determines directly or indirectly most materials characteristics. However, predicting crystal structure of solids remains a formidable and not fully solved problem. Standard theoretical tools for this task are computationally expensive and at times inaccurate. Here we present an alternative approach utilizing machine learning for crystal structure prediction. We developed a tool called Crystal Structure Prediction Network (CRYSPNet) that can predict the Bravais lattice, space group, and lattice parameters of an inorganic material based only on its chemical composition. CRYSPNet consists of a series of neural network models, using as inputs predictors aggregating the properties of the elements constituting the compound. It was trained and validated on more than 100,000 entries from the Inorganic Crystal Structure Database. The tool demonstrates robust predictive capability and outperforms alternative strategies by a large margin. Made available to the public (at https://github.com/AuroraLHT/cryspnet), it can be used both as an independent prediction engine or as a method to generate candidate structures for further computational and/or experimental validation.
Published 2020-03-31
URL https://arxiv.org/abs/2003.14328v1
PDF https://arxiv.org/pdf/2003.14328v1.pdf
PWC https://paperswithcode.com/paper/cryspnet-crystal-structure-predictions-via
Repo https://github.com/AuroraLHT/cryspnet
Framework pytorch

#### NYTWIT: A Dataset of Novel Words in the New York Times

Title NYTWIT: A Dataset of Novel Words in the New York Times
Authors Yuval Pinter, Cassandra L. Jacobs, Max Bittker
Abstract We present the New York Times Word Innovation Types dataset, or NYTWIT, a collection of over 2,500 novel English words published in the New York Times between November 2017 and March 2019, manually annotated for their class of novelty (such as lexical derivation, dialectal variation, blending, or compounding). We present baseline results for both uncontextual and contextual prediction of novelty class, showing that there is room for improvement even for state-of-the-art NLP systems. We hope this resource will prove useful for linguists and NLP practitioners by providing a real-world environment of novel word appearance.
Published 2020-03-06
URL https://arxiv.org/abs/2003.03444v1
PDF https://arxiv.org/pdf/2003.03444v1.pdf
PWC https://paperswithcode.com/paper/nytwit-a-dataset-of-novel-words-in-the-new
Repo https://github.com/yuvalpinter/nytwit
Framework none

#### SAM: The Sensitivity of Attribution Methods to Hyperparameters

Title SAM: The Sensitivity of Attribution Methods to Hyperparameters
Authors Naman Bansal, Chirag Agarwal, Anh Nguyen
Abstract Attribution methods can provide powerful insights into the reasons for a classifier’s decision. We argue that a key desideratum of an explanation method is its robustness to input hyperparameters which are often randomly set or empirically tuned. High sensitivity to arbitrary hyperparameter choices does not only impede reproducibility but also questions the correctness of an explanation and impairs the trust of end-users. In this paper, we provide a thorough empirical study on the sensitivity of existing attribution methods. We found an alarming trend that many methods are highly sensitive to changes in their common hyperparameters e.g. even changing a random seed can yield a different explanation! Interestingly, such sensitivity is not reflected in the average explanation accuracy scores over the dataset as commonly reported in the literature. In addition, explanations generated for robust classifiers (i.e. which are trained to be invariant to pixel-wise perturbations) are surprisingly more robust than those generated for regular classifiers.
Published 2020-03-04
URL https://arxiv.org/abs/2003.08754v1
PDF https://arxiv.org/pdf/2003.08754v1.pdf
Repo https://github.com/anguyen8/sam
Framework pytorch

#### On the distance between two neural networks and the stability of learning

Title On the distance between two neural networks and the stability of learning
Authors Jeremy Bernstein, Arash Vahdat, Yisong Yue, Ming-Yu Liu
Abstract How far apart are two neural networks? This is a foundational question in their theory. We derive a simple and tractable bound that relates distance in function space to distance in parameter space for a broad class of nonlinear compositional functions. The bound distills a clear dependence on depth of the composition. The theory is of practical relevance since it establishes a trust region for first-order optimisation. In turn, this suggests an optimiser that we call Frobenius matched gradient descent—or Fromage. Fromage involves a principled form of gradient rescaling and enjoys guarantees on stability of both the spectra and Frobenius norms of the weights. We find that the new algorithm increases the depth at which a multilayer perceptron may be trained as compared to Adam and SGD and is competitive with Adam for training generative adversarial networks. We further verify that Fromage scales up to a language transformer with over $10^8$ parameters. Please find code & reproducibility instructions at: https://github.com/jxbz/fromage.
Published 2020-02-09
URL https://arxiv.org/abs/2002.03432v1
PDF https://arxiv.org/pdf/2002.03432v1.pdf
PWC https://paperswithcode.com/paper/on-the-distance-between-two-neural-networks
Repo https://github.com/jxbz/fromage
Framework pytorch

#### Exploiting Verified Neural Networks via Floating Point Numerical Error

Title Exploiting Verified Neural Networks via Floating Point Numerical Error
Authors Kai Jia, Martin Rinard
Abstract We show how to construct adversarial examples for neural networks with exactly verified robustness against $\ell_{\infty}$-bounded input perturbations by exploiting floating point error. We argue that any exact verification of real-valued neural networks must accurately model the implementation details of any floating point arithmetic used during inference or verification.
Published 2020-03-06
URL https://arxiv.org/abs/2003.03021v1
PDF https://arxiv.org/pdf/2003.03021v1.pdf
PWC https://paperswithcode.com/paper/exploiting-verified-neural-networks-via
Framework pytorch

#### Deep differentiable forest with sparse attention for the tabular data

Title Deep differentiable forest with sparse attention for the tabular data
Authors Yingshi Chen
Abstract We present a general architecture of deep differentiable forest and its sparse attention mechanism. The differentiable forest has the advantages of both trees and neural networks. Its structure is a simple binary tree, easy to use and understand. It has full differentiability and all variables are learnable parameters. We would train it by the gradient-based optimization method, which shows great power in the training of deep CNN. We find and analyze the attention mechanism in the differentiable forest. That is, each decision depends on only a few important features, and others are irrelevant. The attention is always sparse. Based on this observation, we improve its sparsity by data-aware initialization. We use the attribute importance to initialize the attention weight. Then the learned weight is much sparse than that from random initialization. Our experiment on some large tabular dataset shows differentiable forest has higher accuracy than GBDT, which is the state of art algorithm for tabular datasets. The source codes are available at https://github.com/closest-git/QuantumForest
Published 2020-02-29
URL https://arxiv.org/abs/2003.00223v1
PDF https://arxiv.org/pdf/2003.00223v1.pdf
PWC https://paperswithcode.com/paper/deep-differentiable-forest-with-sparse
Repo https://github.com/closest-git/QuantumForest
Framework pytorch

#### COVID-19 Image Data Collection

Title COVID-19 Image Data Collection
Authors Joseph Paul Cohen, Paul Morrison, Lan Dao
Abstract This paper describes the initial COVID-19 open image data collection. It was created by assembling medical images from websites and publications and currently contains 123 frontal view X-rays.
Published 2020-03-25
URL https://arxiv.org/abs/2003.11597v1
PDF https://arxiv.org/pdf/2003.11597v1.pdf
PWC https://paperswithcode.com/paper/covid-19-image-data-collection
Repo https://github.com/ieee8023/covid-chestxray-dataset
Framework none

#### How Useful is Self-Supervised Pretraining for Visual Tasks?

Title How Useful is Self-Supervised Pretraining for Visual Tasks?
Authors Alejandro Newell, Jia Deng
Abstract Recent advances have spurred incredible progress in self-supervised pretraining for vision. We investigate what factors may play a role in the utility of these pretraining methods for practitioners. To do this, we evaluate various self-supervised algorithms across a comprehensive array of synthetic datasets and downstream tasks. We prepare a suite of synthetic data that enables an endless supply of annotated images as well as full control over dataset difficulty. Our experiments offer insights into how the utility of self-supervision changes as the number of available labels grows as well as how the utility changes as a function of the downstream task and the properties of the training data. We also find that linear evaluation does not correlate with finetuning performance. Code and data is available at \href{https://www.github.com/princeton-vl/selfstudy}{github.com/princeton-vl/selfstudy}.
Published 2020-03-31
URL https://arxiv.org/abs/2003.14323v1
PDF https://arxiv.org/pdf/2003.14323v1.pdf
PWC https://paperswithcode.com/paper/how-useful-is-self-supervised-pretraining-for
Repo https://github.com/princeton-vl/selfstudy
Framework none

#### SuPer Deep: A Surgical Perception Framework for Robotic Tissue Manipulation using Deep Learning for Feature Extraction

Title SuPer Deep: A Surgical Perception Framework for Robotic Tissue Manipulation using Deep Learning for Feature Extraction
Authors Jingpei Lu, Ambareesh Jayakumari, Florian Richter, Yang Li, Michael C. Yip
Abstract Robotic automation in surgery requires precise tracking of surgical tools and mapping of deformable tissue. Previous works on surgical perception frameworks require significant effort in developing features for surgical tool and tissue tracking. In this work, we overcome the challenge by exploiting deep learning methods for surgical perception. We integrated deep neural networks, capable of efficient feature extraction, into the tissue reconstruction and instrument pose estimation processes. By leveraging transfer learning, the deep learning based approach requires minimal training data and reduced feature engineering efforts to fully perceive a surgical scene. The framework was tested on three publicly available datasets, which use the da Vinci Surgical System, for comprehensive analysis. Experimental results show that our framework achieves state-of-the-art tracking performance in a surgical environment by utilizing deep learning for feature extraction.
Tasks Feature Engineering, Pose Estimation, Transfer Learning
Published 2020-03-07
URL https://arxiv.org/abs/2003.03472v1
PDF https://arxiv.org/pdf/2003.03472v1.pdf
PWC https://paperswithcode.com/paper/super-deep-a-surgical-perception-framework
Repo https://github.com/jingpeilu/psmnet_ros
Framework pytorch

#### Represented Value Function Approach for Large Scale Multi Agent Reinforcement Learning

Title Represented Value Function Approach for Large Scale Multi Agent Reinforcement Learning
Authors Weiya Ren
Abstract In this paper, we consider the problem of large scale multi agent reinforcement learning. Firstly, we studied the representation problem of the pairwise value function to reduce the complexity of the interactions among agents. Secondly, we adopt a l2-norm trick to ensure the trivial term of the approximated value function is bounded. Thirdly, experimental results on battle game demonstrate the effectiveness of the proposed approach.
Published 2020-01-04
URL https://arxiv.org/abs/2001.01096v2
PDF https://arxiv.org/pdf/2001.01096v2.pdf
PWC https://paperswithcode.com/paper/represented-value-function-approach-for-large
Repo https://github.com/renweiya/RFQ-RFAC-Represented-Value-Function-Approach-for-Large-Scale-Multi-Agent-Reinforcement-Learning
Framework tf

#### HybridPose: 6D Object Pose Estimation under Hybrid Representations

Title HybridPose: 6D Object Pose Estimation under Hybrid Representations
Authors Chen Song, Jiaru Song, Qixing Huang
Abstract We introduce HybridPose, a novel 6D object pose estimation approach. HybridPose utilizes a hybrid intermediate representation to express different geometric information in the input image, including keypoints, edge vectors, and symmetry correspondences. Compared to a unitary representation, our hybrid representation allows pose regression to exploit more and diverse features when one type of predicted representation is inaccurate (e.g., because of occlusion). Different intermediate representations used by HybridPose can all be predicted by the same simple neural network, and outliers in predicted intermediate representations are filtered by a robust regression module. Compared to state-of-the-art pose estimation approaches, HybridPose is comparable in running time and is significantly more accurate. For example, on Occlusion Linemod dataset, our method achieves a prediction speed of 30 fps with a mean ADD(-S) accuracy of 79.2%, representing a 67.4% improvement from the current state-of-the-art approach. The implementation of HybridPose is available at https://github.com/chensong1995/HybridPose.
Tasks 6D Pose Estimation using RGB, Pose Estimation
Published 2020-01-07
URL https://arxiv.org/abs/2001.01869v2
PDF https://arxiv.org/pdf/2001.01869v2.pdf
PWC https://paperswithcode.com/paper/hybridpose-6d-object-pose-estimation-under
Repo https://github.com/chensong1995/HybridPose
Framework pytorch