October 20, 2019

3075 words 15 mins read

Paper Group AWR 178

Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization with Medical Applications. Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code. A Guide to Constraining Effective Field Theories with Machine Learning. Deep learning incorporating biologically-inspired neural dynamics. Grounded Textu …

Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization with Medical Applications


Title	Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization with Medical Applications
Authors	Geir Thore Berge, Ole-Christoffer Granmo, Tor Oddbjørn Tveit, Morten Goodwin, Lei Jiao, Bernt Viggo Matheussen
Abstract	Medical applications challenge today’s text categorization techniques by demanding both high accuracy and ease-of-interpretation. Although deep learning has provided a leap ahead in accuracy, this leap comes at the sacrifice of interpretability. To address this accuracy-interpretability challenge, we here introduce, for the first time, a text categorization approach that leverages the recently introduced Tsetlin Machine. In all brevity, we represent the terms of a text as propositional variables. From these, we capture categories using simple propositional formulae, such as: if “rash” and “reaction” and “penicillin” then Allergy. The Tsetlin Machine learns these formulae from a labelled text, utilizing conjunctive clauses to represent the particular facets of each category. Indeed, even the absence of terms (negated features) can be used for categorization purposes. Our empirical comparison with Na"ive Bayes, decision trees, linear support vector machines (SVMs), random forest, long short-term memory (LSTM) neural networks, and other techniques, is quite conclusive. The Tsetlin Machine either performs on par with or outperforms all of the evaluated methods on both the 20 Newsgroups and IMDb datasets, as well as on a non-public clinical dataset. On average, the Tsetlin Machine delivers the best recall and precision scores across the datasets. Finally, our GPU implementation of the Tsetlin Machine executes 5 to 15 times faster than the CPU implementation, depending on the dataset. We thus believe that our novel approach can have a significant impact on a wide range of text analysis applications, forming a promising starting point for deeper natural language understanding with the Tsetlin Machine.
Tasks	Text Categorization
Published	2018-09-12
URL	http://arxiv.org/abs/1809.04547v2
PDF	http://arxiv.org/pdf/1809.04547v2.pdf
PWC	https://paperswithcode.com/paper/using-the-tsetlin-machine-to-learn-human
Repo	https://github.com/cair/TextUnderstandingTsetlinMachine
Framework	tf

Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code


Title	Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code
Authors	Riyadh Baghdadi, Jessica Ray, Malek Ben Romdhane, Emanuele Del Sozzo, Abdurrahman Akkas, Yunming Zhang, Patricia Suriana, Shoaib Kamil, Saman Amarasinghe
Abstract	This paper introduces Tiramisu, a polyhedral framework designed to generate high performance code for multiple platforms including multicores, GPUs, and distributed machines. Tiramisu introduces a scheduling language with novel extensions to explicitly manage the complexities that arise when targeting these systems. The framework is designed for the areas of image processing, stencils, linear algebra and deep learning. Tiramisu has two main features: it relies on a flexible representation based on the polyhedral model and it has a rich scheduling language allowing fine-grained control of optimizations. Tiramisu uses a four-level intermediate representation that allows full separation between the algorithms, loop transformations, data layouts, and communication. This separation simplifies targeting multiple hardware architectures with the same algorithm. We evaluate Tiramisu by writing a set of image processing, deep learning, and linear algebra benchmarks and compare them with state-of-the-art compilers and hand-tuned libraries. We show that Tiramisu matches or outperforms existing compilers and libraries on different hardware architectures, including multicore CPUs, GPUs, and distributed machines.
Tasks
Published	2018-04-27
URL	http://arxiv.org/abs/1804.10694v5
PDF	http://arxiv.org/pdf/1804.10694v5.pdf
PWC	https://paperswithcode.com/paper/tiramisu-a-polyhedral-compiler-for-expressing
Repo	https://github.com/Tiramisu-Compiler/tiramisu
Framework	none

A Guide to Constraining Effective Field Theories with Machine Learning


Title	A Guide to Constraining Effective Field Theories with Machine Learning
Authors	Johann Brehmer, Kyle Cranmer, Gilles Louppe, Juan Pavez
Abstract	We develop, discuss, and compare several inference techniques to constrain theory parameters in collider experiments. By harnessing the latent-space structure of particle physics processes, we extract extra information from the simulator. This augmented data can be used to train neural networks that precisely estimate the likelihood ratio. The new methods scale well to many observables and high-dimensional parameter spaces, do not require any approximations of the parton shower and detector response, and can be evaluated in microseconds. Using weak-boson-fusion Higgs production as an example process, we compare the performance of several techniques. The best results are found for likelihood ratio estimators trained with extra information about the score, the gradient of the log likelihood function with respect to the theory parameters. The score also provides sufficient statistics that contain all the information needed for inference in the neighborhood of the Standard Model. These methods enable us to put significantly stronger bounds on effective dimension-six operators than the traditional approach based on histograms. They also outperform generic machine learning methods that do not make use of the particle physics structure, demonstrating their potential to substantially improve the new physics reach of the LHC legacy results.
Tasks
Published	2018-04-30
URL	http://arxiv.org/abs/1805.00020v4
PDF	http://arxiv.org/pdf/1805.00020v4.pdf
PWC	https://paperswithcode.com/paper/a-guide-to-constraining-effective-field
Repo	https://github.com/johannbrehmer/simulator-mining-example
Framework	none

Deep learning incorporating biologically-inspired neural dynamics


Title	Deep learning incorporating biologically-inspired neural dynamics
Authors	Stanisław Woźniak, Angeliki Pantazi, Thomas Bohnstingl, Evangelos Eleftheriou
Abstract	Neural networks have become the key technology of artificial intelligence and have contributed to breakthroughs in several machine learning tasks, primarily owing to advances in deep learning applied to Artificial Neural Networks (ANNs). Simultaneously, Spiking Neural Networks (SNNs) incorporating biologically-feasible spiking neurons have held great promise because of their rich temporal dynamics and high-power efficiency. However, the developments in SNNs were proceeding separately from those in ANNs, effectively limiting the adoption of deep learning research insights. Here we show an alternative perspective on the spiking neuron that casts it as a particular ANN construct called Spiking Neural Unit (SNU), and a soft SNU (sSNU) variant that generalizes its dynamics to a novel recurrent ANN unit. SNUs bridge the biologically-inspired SNNs with ANNs and provide a methodology for seamless inclusion of spiking neurons in deep learning architectures. Furthermore, SNU enables highly-efficient in-memory acceleration of SNNs trained with backpropagation through time, implemented with the hardware in-the-loop. We apply SNUs to tasks ranging from hand-written digit recognition, language modelling, to music prediction. We obtain accuracy comparable to, or better than, that of state-of-the-art ANNs, and we experimentally verify the efficacy of the in-memory-based SNN realization for the music-prediction task using 52,800 phase-change memory devices. The new generation of neural units introduced in this paper incorporate biologically-inspired neural dynamics in deep learning. In addition, they provide a systematic methodology for training neuromorphic computing hardware. Thus, they open a new avenue for a widespread adoption of SNNs in practical applications.
Tasks	Language Modelling
Published	2018-12-17
URL	https://arxiv.org/abs/1812.07040v2
PDF	https://arxiv.org/pdf/1812.07040v2.pdf
PWC	https://paperswithcode.com/paper/deep-networks-incorporating-spiking-neural
Repo	https://github.com/takyamamoto/SNU_Chainer
Framework	none

Grounded Textual Entailment


Title	Grounded Textual Entailment
Authors	Hoa Trong Vu, Claudio Greco, Aliia Erofeeva, Somayeh Jafaritazehjan, Guido Linders, Marc Tanti, Alberto Testoni, Raffaella Bernardi, Albert Gatt
Abstract	Capturing semantic relations between sentences, such as entailment, is a long-standing challenge for computational semantics. Logic-based models analyse entailment in terms of possible worlds (interpretations, or situations) where a premise P entails a hypothesis H iff in all worlds where P is true, H is also true. Statistical models view this relationship probabilistically, addressing it in terms of whether a human would likely infer H from P. In this paper, we wish to bridge these two perspectives, by arguing for a visually-grounded version of the Textual Entailment task. Specifically, we ask whether models can perform better if, in addition to P and H, there is also an image (corresponding to the relevant “world” or “situation”). We use a multimodal version of the SNLI dataset (Bowman et al., 2015) and we compare “blind” and visually-augmented models of textual entailment. We show that visual information is beneficial, but we also conduct an in-depth error analysis that reveals that current multimodal models are not performing “grounding” in an optimal fashion.
Tasks	Natural Language Inference
Published	2018-06-14
URL	http://arxiv.org/abs/1806.05645v1
PDF	http://arxiv.org/pdf/1806.05645v1.pdf
PWC	https://paperswithcode.com/paper/grounded-textual-entailment
Repo	https://github.com/claudiogreco/coling18-gte
Framework	tf

Neural Word Search in Historical Manuscript Collections


Title	Neural Word Search in Historical Manuscript Collections
Authors	Tomas Wilkinson, Jonas Lindström, Anders Brun
Abstract	We address the problem of segmenting and retrieving word images in collections of historical manuscripts given a text query. This is commonly referred to as “word spotting”. To this end, we first propose an end-to-end trainable model based on deep neural networks that we dub Ctrl-F-Net. The model simultaneously generates region proposals and embeds them into a word embedding space, wherein a search is performed. We further introduce a simplified version called Ctrl-F-Mini. It is faster with similar performance, though it is limited to more easily segmented manuscripts. We evaluate both models on common benchmark datasets and surpass the previous state of the art. Finally, in collaboration with historians, we employ the Ctrl-F-Net to search within a large manuscript collection of over 100 thousand pages, written across two centuries. With only 11 training pages, we enable large scale data collection in manuscript-based historical research. This results in a speed up of data collection and the number of manuscripts processed by orders of magnitude. Given the time consuming manual work required to study old manuscripts in the humanities, quick and robust tools for word spotting has the potential to revolutionise domains like history, religion and language.
Tasks
Published	2018-12-06
URL	https://arxiv.org/abs/1812.02771v2
PDF	https://arxiv.org/pdf/1812.02771v2.pdf
PWC	https://paperswithcode.com/paper/neural-word-search-in-historical-manuscript
Repo	https://github.com/tomfalainen/neural-word-search
Framework	pytorch

Capsule Networks against Medical Imaging Data Challenges


Title	Capsule Networks against Medical Imaging Data Challenges
Authors	Amelia Jiménez-Sánchez, Shadi Albarqouni, Diana Mateus
Abstract	A key component to the success of deep learning is the availability of massive amounts of training data. Building and annotating large datasets for solving medical image classification problems is today a bottleneck for many applications. Recently, capsule networks were proposed to deal with shortcomings of Convolutional Neural Networks (ConvNets). In this work, we compare the behavior of capsule networks against ConvNets under typical datasets constraints of medical image analysis, namely, small amounts of annotated data and class-imbalance. We evaluate our experiments on MNIST, Fashion-MNIST and medical (histological and retina images) publicly available datasets. Our results suggest that capsule networks can be trained with less amount of data for the same or better performance and are more robust to an imbalanced class distribution, which makes our approach very promising for the medical imaging community.
Tasks	Image Classification
Published	2018-07-19
URL	http://arxiv.org/abs/1807.07559v1
PDF	http://arxiv.org/pdf/1807.07559v1.pdf
PWC	https://paperswithcode.com/paper/capsule-networks-against-medical-imaging-data
Repo	https://github.com/ameliajimenez/capsule-networks-medical-data-challenges
Framework	tf

Deep Weighted Averaging Classifiers


Title	Deep Weighted Averaging Classifiers
Authors	Dallas Card, Michael Zhang, Noah A. Smith
Abstract	Recent advances in deep learning have achieved impressive gains in classification accuracy on a variety of types of data, including images and text. Despite these gains, however, concerns have been raised about the calibration, robustness, and interpretability of these models. In this paper we propose a simple way to modify any conventional deep architecture to automatically provide more transparent explanations for classification decisions, as well as an intuitive notion of the credibility of each prediction. Specifically, we draw on ideas from nonparametric kernel regression, and propose to predict labels based on a weighted sum of training instances, where the weights are determined by distance in a learned instance-embedding space. Working within the framework of conformal methods, we propose a new measure of nonconformity suggested by our model, and experimentally validate the accompanying theoretical expectations, demonstrating improved transparency, controlled error rates, and robustness to out-of-domain data, without compromising on accuracy or calibration.
Tasks	Calibration
Published	2018-11-06
URL	http://arxiv.org/abs/1811.02579v2
PDF	http://arxiv.org/pdf/1811.02579v2.pdf
PWC	https://paperswithcode.com/paper/deep-weighted-averaging-classifiers
Repo	https://github.com/dallascard/DWAC
Framework	pytorch

Beetle Swarm Optimization Algorithm:Theory and Application


Title	Beetle Swarm Optimization Algorithm:Theory and Application
Authors	Tiantian Wang, Long Yang, Qiang Liu
Abstract	In this paper, a new meta-heuristic algorithm, called beetle swarm optimization algorithm, is proposed by enhancing the performance of swarm optimization through beetle foraging principles. The performance of 23 benchmark functions is tested and compared with widely used algorithms, including particle swarm optimization algorithm, genetic algorithm (GA) and grasshopper optimization algorithm . Numerical experiments show that the beetle swarm optimization algorithm outperforms its counterparts. Besides, to demonstrate the practical impact of the proposed algorithm, two classic engineering design problems, namely, pressure vessel design problem and himmelblaus optimization problem, are also considered and the proposed beetle swarm optimization algorithm is shown to be competitive in those applications.
Tasks
Published	2018-08-01
URL	http://arxiv.org/abs/1808.00206v1
PDF	http://arxiv.org/pdf/1808.00206v1.pdf
PWC	https://paperswithcode.com/paper/beetle-swarm-optimization-algorithmtheory-and
Repo	https://github.com/jywang2016/rBAS
Framework	none

BSAS: Beetle Swarm Antennae Search Algorithm for Optimization Problems


Title	BSAS: Beetle Swarm Antennae Search Algorithm for Optimization Problems
Authors	Jiangyu Wang, Huanxin Chen
Abstract	Beetle antennae search (BAS) is an efficient meta-heuristic algorithm. However, the convergent results of BAS rely heavily on the random beetle direction in every iterations. More specifically, different random seeds may cause different optimized results. Besides, the step-size update algorithm of BAS cannot guarantee objective become smaller in iterative process. In order to solve these problems, this paper proposes Beetle Swarm Antennae Search Algorithm (BSAS) which combines swarm intelligence algorithm with feedback-based step-size update strategy. BSAS employs k beetles to find more optimal position in each moving rather than one beetle. The step-size updates only when k beetles return without better choices. Experiments are carried out on building system identification. The results reveal the efficacy of the BSAS algorithm to avoid influence of random direction of Beetle. In addition, the estimation errors decrease as the beetles number goes up.
Tasks
Published	2018-07-27
URL	http://arxiv.org/abs/1807.10470v1
PDF	http://arxiv.org/pdf/1807.10470v1.pdf
PWC	https://paperswithcode.com/paper/bsas-beetle-swarm-antennae-search-algorithm
Repo	https://github.com/jywang2016/rBAS
Framework	none

Beyond Pham’s algorithm for joint diagonalization


Title	Beyond Pham’s algorithm for joint diagonalization
Authors	Pierre Ablin, Jean-François Cardoso, Alexandre Gramfort
Abstract	The approximate joint diagonalization of a set of matrices consists in finding a basis in which these matrices are as diagonal as possible. This problem naturally appears in several statistical learning tasks such as blind signal separation. We consider the diagonalization criterion studied in a seminal paper by Pham (2001), and propose a new quasi-Newton method for its optimization. Through numerical experiments on simulated and real datasets, we show that the proposed method outper-forms Pham’s algorithm. An open source Python package is released.
Tasks
Published	2018-11-28
URL	http://arxiv.org/abs/1811.11433v1
PDF	http://arxiv.org/pdf/1811.11433v1.pdf
PWC	https://paperswithcode.com/paper/beyond-phams-algorithm-for-joint
Repo	https://github.com/pierreablin/qndiag
Framework	none

Scalable Algorithms for Learning High-Dimensional Linear Mixed Models


Title	Scalable Algorithms for Learning High-Dimensional Linear Mixed Models
Authors	Zilong Tan, Kimberly Roche, Xiang Zhou, Sayan Mukherjee
Abstract	Linear mixed models (LMMs) are used extensively to model dependecies of observations in linear regression and are used extensively in many application areas. Parameter estimation for LMMs can be computationally prohibitive on big data. State-of-the-art learning algorithms require computational complexity which depends at least linearly on the dimension $p$ of the covariates, and often use heuristics that do not offer theoretical guarantees. We present scalable algorithms for learning high-dimensional LMMs with sublinear computational complexity dependence on $p$. Key to our approach are novel dual estimators which use only kernel functions of the data, and fast computational techniques based on the subsampled randomized Hadamard transform. We provide theoretical guarantees for our learning algorithms, demonstrating the robustness of parameter estimation. Finally, we complement the theory with experiments on large synthetic and real data.
Tasks
Published	2018-03-12
URL	http://arxiv.org/abs/1803.04431v1
PDF	http://arxiv.org/pdf/1803.04431v1.pdf
PWC	https://paperswithcode.com/paper/scalable-algorithms-for-learning-high
Repo	https://github.com/ZilongTan/arLMM
Framework	none

Binary Classification in Unstructured Space With Hypergraph Case-Based Reasoning


Title	Binary Classification in Unstructured Space With Hypergraph Case-Based Reasoning
Authors	Alexandre Quemy
Abstract	Binary classification is one of the most common problem in machine learning. It consists in predicting whether a given element belongs to a particular class. In this paper, a new algorithm for binary classification is proposed using a hypergraph representation. The method is agnostic to data representation, can work with multiple data sources or in non-metric spaces, and accommodates with missing values. As a result, it drastically reduces the need for data preprocessing or feature engineering. Each element to be classified is partitioned according to its interactions with the training set. For each class, a seminorm over the training set partition is learnt to represent the distribution of evidence supporting this class. Empirical validation demonstrates its high potential on a wide range of well-known datasets and the results are compared to the state-of-the-art. The time complexity is given and empirically validated. Its robustness with regard to hyperparameter sensitivity is studied and compared to standard classification methods. Finally, the limitation of the model space is discussed, and some potential solutions proposed.
Tasks	Feature Engineering
Published	2018-06-16
URL	http://arxiv.org/abs/1806.06232v3
PDF	http://arxiv.org/pdf/1806.06232v3.pdf
PWC	https://paperswithcode.com/paper/binary-classification-in-unstructured-space
Repo	https://github.com/aquemy/HCBR
Framework	none

Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research


Title	Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
Authors	Matthias Plappert, Marcin Andrychowicz, Alex Ray, Bob McGrew, Bowen Baker, Glenn Powell, Jonas Schneider, Josh Tobin, Maciek Chociej, Peter Welinder, Vikash Kumar, Wojciech Zaremba
Abstract	The purpose of this technical report is two-fold. First of all, it introduces a suite of challenging continuous control tasks (integrated with OpenAI Gym) based on currently existing robotics hardware. The tasks include pushing, sliding and pick & place with a Fetch robotic arm as well as in-hand object manipulation with a Shadow Dexterous Hand. All tasks have sparse binary rewards and follow a Multi-Goal Reinforcement Learning (RL) framework in which an agent is told what to do using an additional input. The second part of the paper presents a set of concrete research ideas for improving RL algorithms, most of which are related to Multi-Goal RL and Hindsight Experience Replay.
Tasks	Continuous Control, Multi-Goal Reinforcement Learning
Published	2018-02-26
URL	http://arxiv.org/abs/1802.09464v2
PDF	http://arxiv.org/pdf/1802.09464v2.pdf
PWC	https://paperswithcode.com/paper/multi-goal-reinforcement-learning-challenging
Repo	https://github.com/eduazv/Open-AI-GYN
Framework	tf

A Pragmatic Guide to Geoparsing Evaluation


Title	A Pragmatic Guide to Geoparsing Evaluation
Authors	Milan Gritta, Mohammad Taher Pilehvar, Nigel Collier
Abstract	Empirical methods in geoparsing have thus far lacked a standard evaluation framework describing the task, metrics and data used to compare state-of-the-art systems. Evaluation is further made inconsistent, even unrepresentative of real-world usage by the lack of distinction between the different types of toponyms, which necessitates new guidelines, a consolidation of metrics and a detailed toponym taxonomy with implications for Named Entity Recognition (NER) and beyond. To address these deficiencies, our manuscript introduces a new framework in three parts. Part 1) Task Definition: clarified via corpus linguistic analysis proposing a fine-grained Pragmatic Taxonomy of Toponyms. Part 2) Metrics: discussed and reviewed for a rigorous evaluation including recommendations for NER/Geoparsing practitioners. Part 3) Evaluation Data: shared via a new dataset called GeoWebNews to provide test/train examples and enable immediate use of our contributions. In addition to fine-grained Geotagging and Toponym Resolution (Geocoding), this dataset is also suitable for prototyping and evaluating machine learning NLP models.
Tasks	Named Entity Recognition
Published	2018-10-29
URL	https://arxiv.org/abs/1810.12368v5
PDF	https://arxiv.org/pdf/1810.12368v5.pdf
PWC	https://paperswithcode.com/paper/a-pragmatic-guide-to-geoparsing-evaluation
Repo	https://github.com/milangritta/Pragmatic-Guide-to-Geoparsing-Evaluation
Framework	none