January 29, 2020

3096 words 15 mins read

Paper Group ANR 627

ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension. Nearly-Unsupervised Hashcode Representations for Relation Extraction. Experimenting with Constraint Programming on GPU. Solving a Flowshop Scheduling Problem with Answer Set Programming: Exploiting the Problem to Reduce the Number of Combinations. Imitatio …

ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension


Title	ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension
Authors	Dheeru Dua, Ananth Gottumukkala, Alon Talmor, Sameer Singh, Matt Gardner
Abstract	Reading comprehension is one of the crucial tasks for furthering research in natural language understanding. A lot of diverse reading comprehension datasets have recently been introduced to study various phenomena in natural language, ranging from simple paraphrase matching and entity typing to entity tracking and understanding the implications of the context. Given the availability of many such datasets, comprehensive and reliable evaluation is tedious and time-consuming for researchers working on this problem. We present an evaluation server, ORB, that reports performance on seven diverse reading comprehension datasets, encouraging and facilitating testing a single model’s capability in understanding a wide variety of reading phenomena. The evaluation server places no restrictions on how models are trained, so it is a suitable test bed for exploring training paradigms and representation learning for general reading facility. As more suitable datasets are released, they will be added to the evaluation server. We also collect and include synthetic augmentations for these datasets, testing how well models can handle out-of-domain questions.
Tasks	Entity Typing, Machine Reading Comprehension, Reading Comprehension, Representation Learning
Published	2019-12-29
URL	https://arxiv.org/abs/1912.12598v1
PDF	https://arxiv.org/pdf/1912.12598v1.pdf
PWC	https://paperswithcode.com/paper/orb-an-open-reading-benchmark-for
Repo
Framework

Nearly-Unsupervised Hashcode Representations for Relation Extraction


Title	Nearly-Unsupervised Hashcode Representations for Relation Extraction
Authors	Sahil Garg, Aram Galstyan, Greg Ver Steeg, Guillermo Cecchi
Abstract	Recently, kernelized locality sensitive hashcodes have been successfully employed as representations of natural language text, especially showing high relevance to biomedical relation extraction tasks. In this paper, we propose to optimize the hashcode representations in a nearly unsupervised manner, in which we only use data points, but not their class labels, for learning. The optimized hashcode representations are then fed to a supervised classifier following the prior work. This nearly unsupervised approach allows fine-grained optimization of each hash function, which is particularly suitable for building hashcode representations generalizing from a training set to a test set. We empirically evaluate the proposed approach for biomedical relation extraction tasks, obtaining significant accuracy improvements w.r.t. state-of-the-art supervised and semi-supervised approaches.
Tasks	Relation Extraction
Published	2019-09-09
URL	https://arxiv.org/abs/1909.03881v1
PDF	https://arxiv.org/pdf/1909.03881v1.pdf
PWC	https://paperswithcode.com/paper/nearly-unsupervised-hashcode-representations
Repo
Framework

Experimenting with Constraint Programming on GPU


Title	Experimenting with Constraint Programming on GPU
Authors	Fabio Tardivo
Abstract	The focus of my PhD thesis is on exploring parallel approaches to efficiently solve problems modeled by constraints and presenting a new proposal. Current solvers are very advanced; they are carefully designed to effectively manage the high-level problems’ description and include refined strategies to avoid useless work. Despite this, finding a solution can take an unacceptable amount of time. Parallelization can mitigate this problem when the instance of the problem modeled is large, as it happens in real world problems. It is done by propagating constraints in parallel and concurrently exploring different parts of the search space. I am developing on a constraint solver that exploits the many cores available on Graphics Processing Units (GPU) to speed up the search.
Tasks
Published	2019-09-19
URL	https://arxiv.org/abs/1909.09213v1
PDF	https://arxiv.org/pdf/1909.09213v1.pdf
PWC	https://paperswithcode.com/paper/experimenting-with-constraint-programming-on
Repo
Framework

Solving a Flowshop Scheduling Problem with Answer Set Programming: Exploiting the Problem to Reduce the Number of Combinations


Title	Solving a Flowshop Scheduling Problem with Answer Set Programming: Exploiting the Problem to Reduce the Number of Combinations
Authors	Carmen Leticia García-Mata, Pedro Rafael Márquez-Gutiérrez
Abstract	Planning and scheduling have been a central theme of research in computer science. In particular, the simplicity of the theoretical approach of a no-wait flowshop scheduling problem does not allow to perceive the problem complexity at first sight. In this paper the applicability of the Answer Set Programming language is explored for the solution of the Automated Wet-etching scheduling problem in Semiconductor Manufacturing Systems. A method based in ranges is proposed in order to reduce the huge number of combinations.
Tasks
Published	2019-08-01
URL	https://arxiv.org/abs/1908.00183v2
PDF	https://arxiv.org/pdf/1908.00183v2.pdf
PWC	https://paperswithcode.com/paper/solving-a-flowshop-scheduling-problem-with
Repo
Framework

Imitation Learning of Factored Multi-agent Reactive Models


Title	Imitation Learning of Factored Multi-agent Reactive Models
Authors	Michael Teng, Tuan Anh Le, Adam Scibior, Frank Wood
Abstract	We apply recent advances in deep generative modeling to the task of imitation learning from biological agents. Specifically, we apply variations of the variational recurrent neural network model to a multi-agent setting where we learn policies of individual uncoordinated agents acting based on their perceptual inputs and their hidden belief state. We learn stochastic policies for these agents directly from observational data, without constructing a reward function. An inference network learned jointly with the policy allows for efficient inference over the agent’s belief state given a sequence of its current perceptual inputs and the prior actions it performed, which lets us extrapolate observed sequences of behavior into the future while maintaining uncertainty estimates over future trajectories. We test our approach on a dataset of flies interacting in a 2D environment, where we demonstrate better predictive performance than existing approaches which learn deterministic policies with recurrent neural networks. We further show that the uncertainty estimates over future trajectories we obtain are well calibrated, which makes them useful for a variety of downstream processing tasks.
Tasks	Imitation Learning
Published	2019-03-12
URL	http://arxiv.org/abs/1903.04714v1
PDF	http://arxiv.org/pdf/1903.04714v1.pdf
PWC	https://paperswithcode.com/paper/imitation-learning-of-factored-multi-agent
Repo
Framework

Levelling the Playing Field: A Comprehensive Comparison of Visual Place Recognition Approaches under Changing Conditions


Title	Levelling the Playing Field: A Comprehensive Comparison of Visual Place Recognition Approaches under Changing Conditions
Authors	Mubariz Zaffar, Ahmad Khaliq, Shoaib Ehsan, Michael Milford, Klaus McDonald-Maier
Abstract	In recent years there has been significant improvement in the capability of Visual Place Recognition (VPR) methods, building on the success of both hand-crafted and learnt visual features, temporal filtering and usage of semantic scene information. The wide range of approaches and the relatively recent growth in interest in the field has meant that a wide range of datasets and assessment methodologies have been proposed, often with a focus only on precision-recall type metrics, making comparison difficult. In this paper we present a comprehensive approach to evaluating the performance of 10 state-of-the-art recently-developed VPR techniques, which utilizes three standardized metrics: (a) Matching Performance b) Matching Time c) Memory Footprint. Together this analysis provides an up-to-date and widely encompassing snapshot of the various strengths and weaknesses of contemporary approaches to the VPR problem. The aim of this work is to help move this particular research field towards a more mature and unified approach to the problem, enabling better comparison and hence more progress to be made in future research.
Tasks	Visual Place Recognition
Published	2019-03-21
URL	http://arxiv.org/abs/1903.09107v2
PDF	http://arxiv.org/pdf/1903.09107v2.pdf
PWC	https://paperswithcode.com/paper/levelling-the-playing-field-a-comprehensive
Repo
Framework

Difficulty Classification of Mountainbike Downhill Trails utilizing Deep Neural Networks


Title	Difficulty Classification of Mountainbike Downhill Trails utilizing Deep Neural Networks
Authors	Stefan Langer, Robert Müller, Kyrill Schmid, Claudia Linnhoff-Popien
Abstract	The difficulty of mountainbike downhill trails is a subjective perception. However, sports-associations and mountainbike park operators attempt to group trails into different levels of difficulty with scales like the Singletrail-Skala (S0-S5) or colored scales (blue, red, black, …) as proposed by The International Mountain Bicycling Association. Inconsistencies in difficulty grading occur due to the various scales, different people grading the trails, differences in topography, and more. We propose an end-to-end deep learning approach to classify trails into three difficulties easy, medium, and hard by using sensor data. With mbientlab Meta Motion r0.2 sensor units, we record accelerometer- and gyroscope data of one rider on multiple trail segments. A 2D convolutional neural network is trained with a stacked and concatenated representation of the aforementioned data as its input. We run experiments with five different sample- and five different kernel sizes and achieve a maximum Sparse Categorical Accuracy of 0.9097. To the best of our knowledge, this is the first work targeting computational difficulty classification of mountainbike downhill trails.
Tasks
Published	2019-08-05
URL	https://arxiv.org/abs/1908.04390v1
PDF	https://arxiv.org/pdf/1908.04390v1.pdf
PWC	https://paperswithcode.com/paper/difficulty-classification-of-mountainbike
Repo
Framework

Predicting class-imbalanced business risk using resampling, regularization, and model ensembling algorithms


Title	Predicting class-imbalanced business risk using resampling, regularization, and model ensembling algorithms
Authors	Yan Wang, Xuelei Sherry Ni
Abstract	We aim at developing and improving the imbalanced business risk modeling via jointly using proper evaluation criteria, resampling, cross-validation, classifier regularization, and ensembling techniques. Area Under the Receiver Operating Characteristic Curve (AUC of ROC) is used for model comparison based on 10-fold cross validation. Two undersampling strategies including random undersampling (RUS) and cluster centroid undersampling (CCUS), as well as two oversampling methods including random oversampling (ROS) and Synthetic Minority Oversampling Technique (SMOTE), are applied. Three highly interpretable classifiers, including logistic regression without regularization (LR), L1-regularized LR (L1LR), and decision tree (DT) are implemented. Two ensembling techniques, including Bagging and Boosting, are applied on the DT classifier for further model improvement. The results show that, Boosting on DT by using the oversampled data containing 50% positives via SMOTE is the optimal model and it can achieve AUC, recall, and F1 score valued 0.8633, 0.9260, and 0.8907, respectively.
Tasks
Published	2019-03-13
URL	http://arxiv.org/abs/1903.05535v1
PDF	http://arxiv.org/pdf/1903.05535v1.pdf
PWC	https://paperswithcode.com/paper/predicting-class-imbalanced-business-risk
Repo
Framework

An Enhanced Electrocardiogram Biometric Authentication System Using Machine Learning


Title	An Enhanced Electrocardiogram Biometric Authentication System Using Machine Learning
Authors	Ebrahim Al Alkeem, Song-Kyoo Kim, Chan Yeob Yeun, M. Jamal Zemerly, Kin Poon, Paul D. Yoo
Abstract	Traditional authentication systems use alphanumeric or graphical passwords, or token-based techniques that require “something you know and something you have”. The disadvantages of these systems include the risks of forgetfulness, loss, and theft. To address these shortcomings, biometric authentication is rapidly replacing traditional authentication methods and is becoming a part of everyday life. The electrocardiogram (ECG) is one of the most recent traits considered for biometric purposes. In this work we describe an ECG-based authentication system suitable for security checks and hospital environments. The proposed system will help investigators studying ECG-based biometric authentication techniques to define dataset boundaries and to acquire high-quality training data. We evaluated the performance of the proposed system and found that it could achieve up to the 92 percent identification accuracy. In addition, by applying the Amang ECG (amgecg) toolbox within MATLAB, we investigated the two parameters that directly affect the accuracy of authentication: the ECG slicing time (sliding window) and the sampling time period, and found their optimal values.
Tasks
Published	2019-06-30
URL	https://arxiv.org/abs/1907.00366v2
PDF	https://arxiv.org/pdf/1907.00366v2.pdf
PWC	https://paperswithcode.com/paper/an-enhanced-electrocardiogram-biometric
Repo
Framework

Integrating Whole Context to Sequence-to-sequence Speech Recognition


Title	Integrating Whole Context to Sequence-to-sequence Speech Recognition
Authors	Ye Bai, Jiangyan Yi, Jianhua Tao, Zhengkun Tian, Zhengqi Wen, Shuai Zhang
Abstract	Because an attention based sequence-to-sequence speech (Seq2Seq) recognition model decodes a token sequence in a left-to-right manner, it is non-trivial for the decoder to leverage the whole context of the target sequence. In this paper, we propose a self-attention mechanism based language model called casual cloze completer (COR), which models the left context and the right context simultaneously. Then, we utilize our previously proposed “Learn Spelling from Teachers” approach to integrate the whole context knowledge from COR to the Seq2Seq model. We conduct the experiments on public Chinese dataset AISHELL-1. The experimental results show that leveraging whole context can improve the performance of the Seq2Seq model.
Tasks	Language Modelling, Sequence-To-Sequence Speech Recognition, Speech Recognition
Published	2019-12-04
URL	https://arxiv.org/abs/1912.01777v1
PDF	https://arxiv.org/pdf/1912.01777v1.pdf
PWC	https://paperswithcode.com/paper/integrating-whole-context-to-sequence-to
Repo
Framework

Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition


Title	Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
Authors	Ye Bai, Jiangyan Yi, Jianhua Tao, Zhengkun Tian, Zhengqi Wen
Abstract	Integrating an external language model into a sequence-to-sequence speech recognition system is non-trivial. Previous works utilize linear interpolation or a fusion network to integrate external language models. However, these approaches introduce external components, and increase decoding computation. In this paper, we instead propose a knowledge distillation based training approach to integrating external language models into a sequence-to-sequence model. A recurrent neural network language model, which is trained on large scale external text, generates soft labels to guide the sequence-to-sequence model training. Thus, the language model plays the role of the teacher. This approach does not add any external component to the sequence-to-sequence model during testing. And this approach is flexible to be combined with shallow fusion technique together for decoding. The experiments are conducted on public Chinese datasets AISHELL-1 and CLMAD. Our approach achieves a character error rate of 9.3%, which is relatively reduced by 18.42% compared with the vanilla sequence-to-sequence model.
Tasks	Language Modelling, Sequence-To-Sequence Speech Recognition, Speech Recognition
Published	2019-07-13
URL	https://arxiv.org/abs/1907.06017v1
PDF	https://arxiv.org/pdf/1907.06017v1.pdf
PWC	https://paperswithcode.com/paper/learn-spelling-from-teachers-transferring
Repo
Framework

Playing Atari Ball Games with Hierarchical Reinforcement Learning


Title	Playing Atari Ball Games with Hierarchical Reinforcement Learning
Authors	Hua Huang, Adrian Barbu
Abstract	Human beings are particularly good at reasoning and inference from just a few examples. When facing new tasks, humans will leverage knowledge and skills learned before, and quickly integrate them with the new task. In addition to learning by experimentation, human also learn socio-culturally through instructions and learning by example. In this way humans can learn much faster compared with most current artificial intelligence algorithms in many tasks. In this paper, we test the idea of speeding up machine learning through social learning. We argue that in solving real-world problems, especially when the task is designed by humans, and/or for humans, there are typically instructions from user manuals and/or human experts which give guidelines on how to better accomplish the tasks. We argue that these instructions have tremendous value in designing a reinforcement learning system which can learn in human fashion, and we test the idea by playing the Atari games Tennis and Pong. We experimentally demonstrate that the instructions provide key information about the task, which can be used to decompose the learning task into sub-systems and construct options for the temporally extended planning, and dramatically accelerate the learning process.
Tasks	Atari Games, Hierarchical Reinforcement Learning
Published	2019-09-27
URL	https://arxiv.org/abs/1909.12465v1
PDF	https://arxiv.org/pdf/1909.12465v1.pdf
PWC	https://paperswithcode.com/paper/playing-atari-ball-games-with-hierarchical
Repo
Framework

Pointwise Attention-Based Atrous Convolutional Neural Networks


Title	Pointwise Attention-Based Atrous Convolutional Neural Networks
Authors	Mobina Mahdavi, Fahimeh Fooladgar, Shohreh Kasaei
Abstract	With the rapid progress of deep convolutional neural networks, in almost all robotic applications, the availability of 3D point clouds improves the accuracy of 3D semantic segmentation methods. Rendering of these irregular, unstructured, and unordered 3D points to 2D images from multiple viewpoints imposes some issues such as loss of information due to 3D to 2D projection, discretizing artifacts, and high computational costs. To efficiently deal with a large number of points and incorporate more context of each point, a pointwise attention-based atrous convolutional neural network architecture is proposed. It focuses on salient 3D feature points among all feature maps while considering outstanding contextual information via spatial channel-wise attention modules. The proposed model has been evaluated on the two most important 3D point cloud datasets for the 3D semantic segmentation task. It achieves a reasonable performance compared to state-of-the-art models in terms of accuracy, with a much smaller number of parameters.
Tasks	3D Semantic Segmentation, Semantic Segmentation
Published	2019-12-27
URL	https://arxiv.org/abs/1912.12082v1
PDF	https://arxiv.org/pdf/1912.12082v1.pdf
PWC	https://paperswithcode.com/paper/pointwise-attention-based-atrous
Repo
Framework

Harnessing Structures for Value-Based Planning and Reinforcement Learning


Title	Harnessing Structures for Value-Based Planning and Reinforcement Learning
Authors	Yuzhe Yang, Guo Zhang, Zhi Xu, Dina Katabi
Abstract	Value-based methods constitute a fundamental methodology in planning and deep reinforcement learning (RL). In this paper, we propose to exploit the underlying structures of the state-action value function, i.e., Q function, for both planning and deep RL. In particular, if the underlying system dynamics lead to some global structures of the Q function, one should be capable of inferring the function better by leveraging such structures. Specifically, we investigate the low-rank structure, which widely exists for big data matrices. We verify empirically the existence of low-rank Q functions in the context of control and deep RL tasks. As our key contribution, by leveraging Matrix Estimation (ME) techniques, we propose a general framework to exploit the underlying low-rank structure in Q functions. This leads to a more efficient planning procedure for classical control, and additionally, a simple scheme that can be applied to value-based RL techniques to consistently achieve better performance on “low-rank” tasks. Extensive experiments on control tasks and Atari games confirm the efficacy of our approach.
Tasks	Atari Games
Published	2019-09-26
URL	https://arxiv.org/abs/1909.12255v2
PDF	https://arxiv.org/pdf/1909.12255v2.pdf
PWC	https://paperswithcode.com/paper/harnessing-structures-for-value-based
Repo
Framework

A Numerical Study of the Time of Extinction in a Class of Systems of Spiking Neurons


Title	A Numerical Study of the Time of Extinction in a Class of Systems of Spiking Neurons
Authors	Cecilia Romaro, Fernando Araujo Najman, Morgan André
Abstract	In this paper we present a numerical study of a mathematical model of spiking neurons introduced by Ferrari et al. in an article entitled Phase transition forinfinite systems of spiking neurons. In this model we have a countable number of neurons linked together in a network, each of them having a membrane potential taking value in the integers, and each of them spiking over time at a rate which depends on the membrane potential through some rate function $\phi$. Beside being affected by a spike each neuron can also be affected by leaking. At each of these leak times, which occurs for a given neuron at a fixed rate $\gamma$, the membrane potential of the neuron concerned is spontaneously reset to $0$. A wide variety of versions of this model can be considered by choosing different graph structures for the network and different activation functions. It was rigorously shown that when the graph structure of the network is the one-dimensional lattice with a hard threshold for the activation function, this model presents a phase transition with respect to $\gamma$, and that it also presents a metastable behavior. By the latter we mean that in the sub-critical regime the re-normalized time of extinction converges to an exponential random variable of mean 1. It has also been proven that in the super-critical regime the renormalized time of extinction converges in probability to 1. Here, we investigate numerically a richer class of graph structures and activation functions. Namely we investigate the case of the two dimensional and the three dimensional lattices, as well as the case of a linear function and a sigmoid function for the activation function. We present numerical evidence that the result of metastability in the sub-critical regime holds for these graphs and activation functions as well as the convergence in probability to $1$ in the super-critical regime.
Tasks
Published	2019-11-06
URL	https://arxiv.org/abs/1911.02609v1
PDF	https://arxiv.org/pdf/1911.02609v1.pdf
PWC	https://paperswithcode.com/paper/a-numerical-study-of-the-time-of-extinction
Repo
Framework