January 29, 2020

3342 words 16 mins read

Paper Group ANR 564

Using Neural Networks for Programming by Demonstration. Automatic Frame Selection Using MLP Neural Network in Ultrasound Elastography. Deep Reinforcement Learning for Motion Planning of Mobile Robots. Evaluating the Effectiveness of Margin Parameter when Learning Knowledge Embedding Representation for Domain-specific Multi-relational Categorized Da …

Using Neural Networks for Programming by Demonstration


Title	Using Neural Networks for Programming by Demonstration
Authors	Karan K. Budhraja, Hang Gao, Tim Oates
Abstract	Agent-based modeling is a paradigm of modeling dynamic systems of interacting agents that are individually governed by specified behavioral rules. Training a model of such agents to produce an emergent behavior by specification of the emergent (as opposed to agent) behavior is easier from a demonstration perspective. Without the involvement of manual behavior specification via code or reliance on a defined taxonomy of possible behaviors, the demonstrator specifies the desired emergent behavior of the system over time, and retrieves agent-level parameters required to execute that motion. A low time-complexity and data requirement favoring framework for reproducing emergent behavior, given an abstract demonstration, is discussed in [1], [2]. The existing framework does, however, observe an inherent limitation in scalability because of an exponentially growing search space (with the number of agent-level parameters). Our work addresses this limitation by pursuing a more scalable architecture with the use of neural networks. While the (proof-of-concept) architecture is not suitable for many evaluated domains because of its lack of representational capacity for that domain, it is more suitable than existing work for larger datasets for the Civil Violence agent-based model.
Tasks
Published	2019-10-10
URL	https://arxiv.org/abs/1910.04724v1
PDF	https://arxiv.org/pdf/1910.04724v1.pdf
PWC	https://paperswithcode.com/paper/using-neural-networks-for-programming-by
Repo
Framework

Automatic Frame Selection Using MLP Neural Network in Ultrasound Elastography


Title	Automatic Frame Selection Using MLP Neural Network in Ultrasound Elastography
Authors	Abdelrahman Zayed, Hassan Rivaz
Abstract	Ultrasound elastography estimates the mechanical properties of the tissue from two Radio-Frequency (RF) frames collected before and after tissue deformation due to an external or internal force. This work focuses on strain imaging in quasi-static elastography, where the tissue undergoes slow deformations and strain images are estimated as a surrogate for elasticity modulus. The quality of the strain image depends heavily on the underlying deformation, and even the best strain estimation algorithms cannot estimate a good strain image if the underlying deformation is not suitable. Herein, we introduce a new method for tracking the RF frames and selecting automatically the best possible pair. We achieve this by decomposing the axial displacement image into a linear combination of principal components (which are calculated offline) multiplied by their corresponding weights. We then use the calculated weights as the input feature vector to a multi-layer perceptron (MLP) classifier. The output is a binary decision, either 1 which refers to good frames, or 0 which refers to bad frames. Our MLP model is trained on in-vivo dataset and tested on different datasets of both in-vivo and phantom data. Results show that by using our technique, we would be able to achieve higher quality strain images compared to the traditional methods of picking up pairs that are 1, 2 or 3 frames apart. The training phase of our algorithm is computationally expensive and takes few hours, but it is only done once. The testing phase chooses the optimal pair of frames in only 1.9 ms.
Tasks
Published	2019-11-13
URL	https://arxiv.org/abs/1911.05245v1
PDF	https://arxiv.org/pdf/1911.05245v1.pdf
PWC	https://paperswithcode.com/paper/automatic-frame-selection-using-mlp-neural
Repo
Framework

Deep Reinforcement Learning for Motion Planning of Mobile Robots


Title	Deep Reinforcement Learning for Motion Planning of Mobile Robots
Authors	Leonid Butyrev, Thorsten Edelhäußer, Christopher Mutschler
Abstract	This paper presents a novel motion and trajectory planning algorithm for nonholonomic mobile robots that uses recent advances in deep reinforcement learning. Starting from a random initial state, i.e., position, velocity and orientation, the robot reaches an arbitrary target state while taking both kinematic and dynamic constraints into account. Our deep reinforcement learning agent not only processes a continuous state space it also executes continuous actions, i.e., the acceleration of wheels and the adaptation of the steering angle. We evaluate our motion and trajectory planning on a mobile robot with a differential drive in a simulation environment.
Tasks	Motion Planning
Published	2019-12-19
URL	https://arxiv.org/abs/1912.09260v1
PDF	https://arxiv.org/pdf/1912.09260v1.pdf
PWC	https://paperswithcode.com/paper/deep-reinforcement-learning-for-motion
Repo
Framework

Evaluating the Effectiveness of Margin Parameter when Learning Knowledge Embedding Representation for Domain-specific Multi-relational Categorized Data


Title	Evaluating the Effectiveness of Margin Parameter when Learning Knowledge Embedding Representation for Domain-specific Multi-relational Categorized Data
Authors	Matthew Wai Heng Chung, Hegler Tissot
Abstract	Learning knowledge representation is an increasingly important technology that supports a variety of machine learning related applications. However, the choice of hyperparameters is seldom justified and usually relies on exhaustive search. Understanding the effect of hyperparameter combinations on embedding quality is crucial to avoid the inefficient process and enhance practicality of vector representation methods. We evaluate the effects of distinct values for the margin parameter focused on translational embedding representation models for multi-relational categorized data. We assess the margin influence regarding the quality of embedding models by contrasting traditional link prediction task accuracy against a classification task. The findings provide evidence that lower values of margin are not rigorous enough to help with the learning process, whereas larger values produce much noise pushing the entities beyond to the surface of the hyperspace, thus requiring constant regularization. Finally, the correlation between link prediction and classification accuracy shows traditional validation protocol for embedding models is a weak metric to represent the quality of embedding representation.
Tasks	Link Prediction
Published	2019-12-21
URL	https://arxiv.org/abs/1912.10264v1
PDF	https://arxiv.org/pdf/1912.10264v1.pdf
PWC	https://paperswithcode.com/paper/evaluating-the-effectiveness-of-margin
Repo
Framework

Neural Subgraph Isomorphism Counting


Title	Neural Subgraph Isomorphism Counting
Authors	Xin Liu, Haojie Pan, Mutian He, Yangqiu Song, Xin Jiang
Abstract	In this paper, we study a new graph learning problem: learning to count subgraph isomorphisms. Although the learning based approach is inexact, we are able to generalize to count large patterns and data graphs in polynomial time compared to the exponential time of the original NP-complete problem. Different from other traditional graph learning problems such as node classification and link prediction, subgraph isomorphism counting requires more global inference to oversee the whole graph. To tackle this problem, we propose a dynamic intermedium attention memory network (DIAMNet) which augments different representation learning architectures and iteratively attends pattern and target data graphs to memorize different subgraph isomorphisms for the global counting. We develop both small graphs (<= 1,024 subgraph isomorphisms in each) and large graphs (<= 4,096 subgraph isomorphisms in each) sets to evaluate different models. Experimental results show that learning based subgraph isomorphism counting can help reduce the time complexity with acceptable accuracy. Our DIAMNet can further improve existing representation learning models for this more global problem.
Tasks	Link Prediction, Node Classification, Representation Learning
Published	2019-12-25
URL	https://arxiv.org/abs/1912.11589v1
PDF	https://arxiv.org/pdf/1912.11589v1.pdf
PWC	https://paperswithcode.com/paper/neural-subgraph-isomorphism-counting-1
Repo
Framework

Learning to Find Hydrological Corrections


Title	Learning to Find Hydrological Corrections
Authors	Lars Arge, Allan Grønlund, Svend Christian Svendsen, Jonas Tranberg
Abstract	High resolution Digital Elevation models, such as the (Big) grid terrain model of Denmark with more than 200 billion measurements, is a basic requirement for water flow modelling and flood risk analysis. However, a large number of modifications often need to be made to even very accurate terrain models, such as the Danish model, before they can be used in realistic flow modeling. These modifications include removal of bridges, which otherwise will act as dams in flow modeling, and inclusion of culverts that transport water underneath roads. In fact, the danish model is accompanied by a detailed set of hydrological corrections for the digital elevation model. However, producing these hydrological corrections is a very slow an expensive process, since it is to a large extent done manually and often with local input. This also means that corrections can be of varying quality. In this paper we propose a new algorithmic apporach based on machine learning and convolutional neural networks for automatically detecting hydrological corrections for such large terrain data. Our model is able to detect most hydrological corrections known for the danish model and quite a few more that should have been included in the original list.
Tasks
Published	2019-09-17
URL	https://arxiv.org/abs/1909.07685v1
PDF	https://arxiv.org/pdf/1909.07685v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-find-hydrological-corrections
Repo
Framework

Fast Computation of Katz Index for Efficient Processing of Link Prediction Queries


Title	Fast Computation of Katz Index for Efficient Processing of Link Prediction Queries
Authors	Mustafa Coskun, Abdelkader Baggag, Mehmet Koyuturk
Abstract	Network proximity computations are among the most common operations in various data mining applications, including link prediction and collaborative filtering. A common measure of network proximity is Katz index, which has been shown to be among the best-performing path-based link prediction algorithms. With the emergence of very large network databases, such proximity computations become an important part of query processing in these databases. Consequently, significant effort has been devoted to developing algorithms for efficient computation of Katz index between a given pair of nodes or between a query node and every other node in the network. Here, we present LRC-Katz, an algorithm based on indexing and low-rank correction to accelerate Katz index-based network proximity queries. Using a variety of very large real-world networks, we show that LRC-Katz outperforms the fastest existing method, Conjugate Gradient, for a wide range of parameter values. We also show that this acceleration in the computation of Katz index can be used to drastically improve the efficiency of processing link prediction queries in very large networks. Motivated by this observation, we propose a new link prediction algorithm that exploits modularity of networks that are encountered in practical applications. Our experimental results on the link prediction problem show that our modularity based algorithm significantly outperforms the state-of-the-art link prediction Katz method.
Tasks	Link Prediction
Published	2019-12-13
URL	https://arxiv.org/abs/1912.06525v1
PDF	https://arxiv.org/pdf/1912.06525v1.pdf
PWC	https://paperswithcode.com/paper/fast-computation-of-katz-index-for-efficient
Repo
Framework

Solving Math Word Problems with Double-Decoder Transformer


Title	Solving Math Word Problems with Double-Decoder Transformer
Authors	Yuanliang Meng, Anna Rumshisky
Abstract	This paper proposes a Transformer-based model to generate equations for math word problems. It achieves much better results than RNN models when copy and align mechanisms are not used, and can outperform complex copy and align RNN models. We also show that training a Transformer jointly in a generation task with two decoders, left-to-right and right-to-left, is beneficial. Such a Transformer performs better than the one with just one decoder not only because of the ensemble effect, but also because it improves the encoder training procedure. We also experiment with adding reinforcement learning to our model, showing improved performance compared to MLE training.
Tasks
Published	2019-08-28
URL	https://arxiv.org/abs/1908.10924v1
PDF	https://arxiv.org/pdf/1908.10924v1.pdf
PWC	https://paperswithcode.com/paper/solving-math-word-problems-with-double
Repo
Framework

Risk Aware Ranking for Top-$k$ Recommendations


Title	Risk Aware Ranking for Top-$k$ Recommendations
Authors	Shameem A Puthiya Parambath, Nishant Vijayakumar, Sanjay Chawla
Abstract	Given an incomplete ratings data over a set of users and items, the preference completion problem aims to estimate a personalized total preference order over a subset of the items. In practical settings, a ranked list of top-$k$ items from the estimated preference order is recommended to the end user in the decreasing order of preference for final consumption. We analyze this model and observe that such a ranking model results in suboptimal performance when the payoff associated with the recommended items is different. We propose a novel and very efficient algorithm for the preference ranking considering the uncertainty regarding the payoffs of the items. Once the preference scores for the users are obtained using any preference learning algorithm, we show that ranking the items using a risk seeking utility function results in the best ranking performance.
Tasks
Published	2019-03-17
URL	http://arxiv.org/abs/1904.05325v2
PDF	http://arxiv.org/pdf/1904.05325v2.pdf
PWC	https://paperswithcode.com/paper/risk-aware-reranking-for-top-k
Repo
Framework

Judging Chemical Reaction Practicality From Positive Sample only Learning


Title	Judging Chemical Reaction Practicality From Positive Sample only Learning
Authors	Shu Jiang, Zhuosheng Zhang, Hai Zhao, Jiangtong Li, Yang Yang, Bao-Liang Lu, Ning Xia
Abstract	Chemical reaction practicality is the core task among all symbol intelligence based chemical information processing, for example, it provides indispensable clue for further automatic synthesis route inference. Considering that chemical reactions have been represented in a language form, we propose a new solution to generally judge the practicality of organic reaction without considering complex quantum physical modeling or chemistry knowledge. While tackling the practicality judgment as a machine learning task from positive and negative (chemical reaction) samples, all existing studies have to carefully handle the serious insufficiency issue on the negative samples. We propose an auto-construction method to well solve the extensively existed long-term difficulty. Experimental results show our model can effectively predict the practicality of chemical reactions, which achieves a high accuracy of 99.76% on real large-scale chemical lab reaction practicality judgment.
Tasks
Published	2019-04-22
URL	http://arxiv.org/abs/1904.09824v1
PDF	http://arxiv.org/pdf/1904.09824v1.pdf
PWC	https://paperswithcode.com/paper/judging-chemical-reaction-practicality-from
Repo
Framework

Beyond Node Embedding: A Direct Unsupervised Edge Representation Framework for Homogeneous Networks


Title	Beyond Node Embedding: A Direct Unsupervised Edge Representation Framework for Homogeneous Networks
Authors	Sambaran Bandyopadhyay, Anirban Biswas, M. N. Murty, Ramasuri Narayanam
Abstract	Network representation learning has traditionally been used to find lower dimensional vector representations of the nodes in a network. However, there are very important edge driven mining tasks of interest to the classical network analysis community, which have mostly been unexplored in the network embedding space. For applications such as link prediction in homogeneous networks, vector representation (i.e., embedding) of an edge is derived heuristically just by using simple aggregations of the embeddings of the end vertices of the edge. Clearly, this method of deriving edge embedding is suboptimal and there is a need for a dedicated unsupervised approach for embedding edges by leveraging edge properties of the network. Towards this end, we propose a novel concept of converting a network to its weighted line graph which is ideally suited to find the embedding of edges of the original network. We further derive a novel algorithm to embed the line graph, by introducing the concept of collective homophily. To the best of our knowledge, this is the first direct unsupervised approach for edge embedding in homogeneous information networks, without relying on the node embeddings. We validate the edge embeddings on three downstream edge mining tasks. Our proposed optimization framework for edge embedding also generates a set of node embeddings, which are not just the aggregation of edges. Further experimental analysis shows the connection of our framework to the concept of node centrality.
Tasks	Link Prediction, Network Embedding, Representation Learning
Published	2019-12-11
URL	https://arxiv.org/abs/1912.05140v1
PDF	https://arxiv.org/pdf/1912.05140v1.pdf
PWC	https://paperswithcode.com/paper/beyond-node-embedding-a-direct-unsupervised
Repo
Framework

Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors


Title	Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors
Authors	Qian Yue, Xinzhe Luo, Qing Ye, Lingchao Xu, Xiahai Zhuang
Abstract	Cardiac segmentation from late gadolinium enhancement MRI is an important task in clinics to identify and evaluate the infarction of myocardium. The automatic segmentation is however still challenging, due to the heterogeneous intensity distributions and indistinct boundaries in the images. In this paper, we propose a new method, based on deep neural networks (DNN), for fully automatic segmentation. The proposed network, referred to as SRSCN, comprises a shape reconstruction neural network (SRNN) and a spatial constraint network (SCN). SRNN aims to maintain a realistic shape of the resulting segmentation. It can be pre-trained by a set of label images, and then be embedded into a unified loss function as a regularization term. Hence, no manually designed feature is needed. Furthermore, SCN incorporates the spatial information of the 2D slices. It is formulated and trained with the segmentation network via the multi-task learning strategy. We evaluated the proposed method using 45 patients and compared with two state-of-the-art regularization schemes, i.e., the anatomically constraint neural network and the adversarial neural network. The results show that the proposed SRSCN outperformed the conventional schemes, and obtained a Dice score of 0.758(std=0.227) for myocardial segmentation, which compares with 0.757(std=0.083) from the inter-observer variations.
Tasks	Cardiac Segmentation, Multi-Task Learning
Published	2019-06-18
URL	https://arxiv.org/abs/1906.07347v2
PDF	https://arxiv.org/pdf/1906.07347v2.pdf
PWC	https://paperswithcode.com/paper/cardiac-segmentation-from-lge-mri-using-deep
Repo
Framework

Sparse Graph Attention Networks


Title	Sparse Graph Attention Networks
Authors	Yang Ye, Shihao Ji
Abstract	Graph Neural Networks (GNNs) have proved to be an effective representation learning framework for graph-structured data, and have achieved state-of-the-art performance on all sorts of practical tasks, such as node classification, link prediction and graph classification. Among the variants of GNNs, Graph Attention Networks (GATs) learn to assign dense attention coefficients over all neighbors of a node for feature aggregation, and improve the performance of many graph learning tasks. However, real-world graphs are often very large and noisy, and GATs are plagued to overfitting if not regularized properly. In this paper, we propose Sparse Graph Attention Networks (SGATs) that learn sparse attention coefficients under an $L_0$-norm regularization, and the learned sparse attentions are then used for all GNN layers, resulting in an edge-sparsified graph. By doing so, we can identify noisy / insignificant edges, and thus focus computation on more important portion of a graph. Extensive experiments on synthetic and real-world graph learning benchmarks demonstrate the superior performance of SGATs. In particular, SGATs can remove about 50%-80% edges from large graphs, such as PPI and Reddit, while retaining similar classification accuracies. Furthermore, the removed edges can be interpreted intuitively and quantitatively. To the best of our knowledge, this is the first graph learning algorithm that sparsifies graphs for the purpose of identifying important relationship between nodes and for robust training.
Tasks	Graph Classification, Link Prediction, Node Classification, Representation Learning
Published	2019-12-02
URL	https://arxiv.org/abs/1912.00552v1
PDF	https://arxiv.org/pdf/1912.00552v1.pdf
PWC	https://paperswithcode.com/paper/sparse-graph-attention-networks
Repo
Framework

Mixed Precision Training With 8-bit Floating Point


Title	Mixed Precision Training With 8-bit Floating Point
Authors	Naveen Mellempudi, Sudarshan Srinivasan, Dipankar Das, Bharat Kaul
Abstract	Reduced precision computation for deep neural networks is one of the key areas addressing the widening compute gap driven by an exponential growth in model size. In recent years, deep learning training has largely migrated to 16-bit precision, with significant gains in performance and energy efficiency. However, attempts to train DNNs at 8-bit precision have met with significant challenges because of the higher precision and dynamic range requirements of back-propagation. In this paper, we propose a method to train deep neural networks using 8-bit floating point representation for weights, activations, errors, and gradients. In addition to reducing compute precision, we also reduced the precision requirements for the master copy of weights from 32-bit to 16-bit. We demonstrate state-of-the-art accuracy across multiple data sets (imagenet-1K, WMT16) and a broader set of workloads (Resnet-18/34/50, GNMT, Transformer) than previously reported. We propose an enhanced loss scaling method to augment the reduced subnormal range of 8-bit floating point for improved error propagation. We also examine the impact of quantization noise on generalization and propose a stochastic rounding technique to address gradient noise. As a result of applying all these techniques, we report slightly higher validation accuracy compared to full precision baseline.
Tasks	Quantization
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12334v1
PDF	https://arxiv.org/pdf/1905.12334v1.pdf
PWC	https://paperswithcode.com/paper/mixed-precision-training-with-8-bit-floating
Repo
Framework

Are Noisy Sentences Useless for Distant Supervised Relation Extraction?


Title	Are Noisy Sentences Useless for Distant Supervised Relation Extraction?
Authors	Yuming Shang
Abstract	The noisy labeling problem has been one of the major obstacles for distant supervised relation extraction. Existing approaches usually consider that the noisy sentences are useless and will harm the model’s performance. Therefore, they mainly alleviate this problem by reducing the influence of noisy sentences, such as applying bag-level selective attention or removing noisy sentences from sentence-bags. However, the underlying cause of the noisy labeling problem is not the lack of useful information, but the missing relation labels. Intuitively, if we can allocate credible labels for noisy sentences, they will be transformed into useful training data and benefit the model’s performance. Thus, in this paper, we propose a novel method for distant supervised relation extraction, which employs unsupervised deep clustering to generate reliable labels for noisy sentences. Specifically, our model contains three modules: a sentence encoder, a noise detector and a label generator. The sentence encoder is used to obtain feature representations. The noise detector detects noisy sentences from sentence-bags, and the label generator produces high-confidence relation labels for noisy sentences. Extensive experimental results demonstrate that our model outperforms the state-of-the-art baselines on a popular benchmark dataset, and can indeed alleviate the noisy labeling problem.
Tasks	Relation Extraction
Published	2019-11-22
URL	https://arxiv.org/abs/1911.09788v1
PDF	https://arxiv.org/pdf/1911.09788v1.pdf
PWC	https://paperswithcode.com/paper/are-noisy-sentences-useless-for-distant
Repo
Framework