January 29, 2020

2808 words 14 mins read

Paper Group ANR 549

The Good, the Bad and the Ugly: Augmenting a black-box model with expert knowledge. Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions. The Chan-Vese Model with Elastica and Landmark Constraints for Image Segmentation. BUDA.ART: A Multimodal Content-Based Analysis and Retrieval System for Buddha Statues. Exploring Self-A …

The Good, the Bad and the Ugly: Augmenting a black-box model with expert knowledge


Title	The Good, the Bad and the Ugly: Augmenting a black-box model with expert knowledge
Authors	Raoul Heese, Michał Walczak, Lukas Morand, Dirk Helm, Michael Bortz
Abstract	We address a non-unique parameter fitting problem in the context of material science. In particular, we propose to resolve ambiguities in parameter space by augmenting a black-box artificial neural network (ANN) model with two different levels of expert knowledge and benchmark them against a pure black-box model.
Tasks
Published	2019-07-24
URL	https://arxiv.org/abs/1907.11105v1
PDF	https://arxiv.org/pdf/1907.11105v1.pdf
PWC	https://paperswithcode.com/paper/the-good-the-bad-and-the-ugly-augmenting-a
Repo
Framework

Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions


Title	Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions
Authors	Awni Hannun, Ann Lee, Qiantong Xu, Ronan Collobert
Abstract	We propose a fully convolutional sequence-to-sequence encoder architecture with a simple and efficient decoder. Our model improves WER on LibriSpeech while being an order of magnitude more efficient than a strong RNN baseline. Key to our approach is a time-depth separable convolution block which dramatically reduces the number of parameters in the model while keeping the receptive field large. We also give a stable and efficient beam search inference procedure which allows us to effectively integrate a language model. Coupled with a convolutional language model, our time-depth separable convolution architecture improves by more than 22% relative WER over the best previously reported sequence-to-sequence results on the noisy LibriSpeech test set.
Tasks	Language Modelling, Sequence-To-Sequence Speech Recognition, Speech Recognition
Published	2019-04-04
URL	http://arxiv.org/abs/1904.02619v1
PDF	http://arxiv.org/pdf/1904.02619v1.pdf
PWC	https://paperswithcode.com/paper/sequence-to-sequence-speech-recognition-with
Repo
Framework

The Chan-Vese Model with Elastica and Landmark Constraints for Image Segmentation


Title	The Chan-Vese Model with Elastica and Landmark Constraints for Image Segmentation
Authors	Jintao Song, Huizhu Pan, Wuanquan Liu, Zisen Xu, Zhenkuan Pan
Abstract	In order to completely separate objects with large sections of occluded boundaries in an image, we devise a new variational level set model for image segmentation combining the Chan-Vese model with elastica and landmark constraints. For computational efficiency, we design its Augmented Lagrangian Method (ALM) or Alternating Direction Method of Multiplier (ADMM) method by introducing some auxiliary variables, Lagrange multipliers, and penalty parameters. In each loop of alternating iterative optimization, the sub-problems of minimization can be easily solved via the Gauss-Seidel iterative method and generalized soft thresholding formulas with projection, respectively. Numerical experiments show that the proposed model can not only recover larger broken boundaries but can also improve segmentation efficiency, as well as decrease the dependence of segmentation on parameter tuning and initialization.
Tasks	Semantic Segmentation
Published	2019-05-27
URL	https://arxiv.org/abs/1905.11192v2
PDF	https://arxiv.org/pdf/1905.11192v2.pdf
PWC	https://paperswithcode.com/paper/the-chan-vese-model-with-elastica-and
Repo
Framework

BUDA.ART: A Multimodal Content-Based Analysis and Retrieval System for Buddha Statues


Title	BUDA.ART: A Multimodal Content-Based Analysis and Retrieval System for Buddha Statues
Authors	Benjamin Renoust, Matheus Oliveira Franca, Jacob Chan, Van Le, Ayaka Uesaka, Yuta Nakashima, Hajime Nagahara, Jueren Wang, Yutaka Fujioka
Abstract	We introduce BUDA.ART, a system designed to assist researchers in Art History, to explore and analyze an archive of pictures of Buddha statues. The system combines different CBIR and classical retrieval techniques to assemble 2D pictures, 3D statue scans and meta-data, that is focused on the Buddha facial characteristics. We build the system from an archive of 50,000 Buddhism pictures, identify unique Buddha statues, extract contextual information, and provide specific facial embedding to first index the archive. The system allows for mobile, on-site search, and to explore similarities of statues in the archive. In addition, we provide search visualization and 3D analysis of the statues
Tasks
Published	2019-09-17
URL	https://arxiv.org/abs/1909.12932v1
PDF	https://arxiv.org/pdf/1909.12932v1.pdf
PWC	https://paperswithcode.com/paper/budaart-a-multimodal-content-based-analysis
Repo
Framework

Exploring Self-Assembling Behaviors in a Swarm of Bio-micro-robots using Surrogate-Assisted MAP-Elites


Title	Exploring Self-Assembling Behaviors in a Swarm of Bio-micro-robots using Surrogate-Assisted MAP-Elites
Authors	Leo Cazenille, Nicolas Bredeche, Nathanael Aubert-Kato
Abstract	Swarms of molecular robots are a promising approach to create specific shapes at the microscopic scale through self-assembly. However, controlling their behavior is a challenging problem as it involves complex non-linear dynamics and high experimental variability. Hand-crafting a molecular controller will often be time-consuming and give sub-optimal results. Optimization methods, like the bioNEAT algorithm, were previously employed to partially overcome these difficulties, but they still had to cope with deceptive high-dimensional search spaces and computationally expensive simulations. Here, we describe a novel approach to solve this problem by using MAP-Elites, an algorithm that searches for both high-performing and diverse solutions. We then apply it to a molecular robotic framework we recently introduced that allows sensing, signaling and self-assembly at the micro-scale and show that MAP-Elites outperforms previous approaches. Additionally, we propose a surrogate model of micro-robots physics and chemical reaction dynamics to reduce the computational costs of simulation. We show that the resulting methodology is capable of optimizing controllers with similar accuracy as when using only a full-fledged realistic model, with half the computational budget.
Tasks
Published	2019-10-01
URL	https://arxiv.org/abs/1910.00230v1
PDF	https://arxiv.org/pdf/1910.00230v1.pdf
PWC	https://paperswithcode.com/paper/exploring-self-assembling-behaviors-in-a
Repo
Framework

Lecture Notes: Selected topics on robust statistical learning theory


Title	Lecture Notes: Selected topics on robust statistical learning theory
Authors	Matthieu Lerasle
Abstract	These notes gather recent results on robust statistical learning theory. The goal is to stress the main principles underlying the construction and theoretical analysis of these estimators rather than provide an exhaustive account on this rapidly growing field. The notes are the basis of lectures given at the conference StatMathAppli 2019.
Tasks
Published	2019-08-28
URL	https://arxiv.org/abs/1908.10761v1
PDF	https://arxiv.org/pdf/1908.10761v1.pdf
PWC	https://paperswithcode.com/paper/lecture-notes-selected-topics-on-robust
Repo
Framework

Complex Trainable ISTA for Linear and Nonlinear Inverse Problems


Title	Complex Trainable ISTA for Linear and Nonlinear Inverse Problems
Authors	Satoshi Takabe, Tadashi Wadayama, Yonina C. Eldar
Abstract	Complex-field signal recovery problems from noisy linear/nonlinear measurements appear in many areas of signal processing and wireless communications. In this paper, we propose a trainable iterative signal recovery algorithm named complex-field TISTA (C-TISTA) which treats complex-field nonlinear inverse problems. C-TISTA is based on the concept of deep unfolding and consists of a gradient descent step with the Wirtinger derivatives followed by a shrinkage step with a trainable complex-valued shrinkage function. Importantly, it contains a small number of trainable parameters so that its training process can be executed efficiently. Numerical results indicate that C-TISTA shows remarkable signal recovery performance compared with existing algorithms.
Tasks
Published	2019-04-16
URL	https://arxiv.org/abs/1904.07409v2
PDF	https://arxiv.org/pdf/1904.07409v2.pdf
PWC	https://paperswithcode.com/paper/complex-field-trainable-ista-for-linear-and
Repo
Framework

Random forest model identifies serve strength as a key predictor of tennis match outcome


Title	Random forest model identifies serve strength as a key predictor of tennis match outcome
Authors	Zijian Gao, Amanda Kowalczyk
Abstract	Tennis is a popular sport worldwide, boasting millions of fans and numerous national and international tournaments. Like many sports, tennis has benefitted from the popularity of rigorous record-keeping of game and player information, as well as the growth of machine learning methods for use in sports analytics. Of particular interest to bettors and betting companies alike is potential use of sports records to predict tennis match outcomes prior to match start. We compiled, cleaned, and used the largest database of tennis match information to date to predict match outcome using fairly simple machine learning methods. Using such methods allows for rapid fit and prediction times to readily incorporate new data and make real-time predictions. We were able to predict match outcomes with upwards of 80% accuracy, much greater than predictions using betting odds alone, and identify serve strength as a key predictor of match outcome. By combining prediction accuracies from three models, we were able to nearly recreate a probability distribution based on average betting odds from betting companies, which indicates that betting companies are using similar information to assign odds to matches. These results demonstrate the capability of relatively simple machine learning models to quite accurately predict tennis match outcomes.
Tasks
Published	2019-10-08
URL	https://arxiv.org/abs/1910.03203v1
PDF	https://arxiv.org/pdf/1910.03203v1.pdf
PWC	https://paperswithcode.com/paper/random-forest-model-identifies-serve-strength
Repo
Framework

Pragmatic inference and visual abstraction enable contextual flexibility during visual communication


Title	Pragmatic inference and visual abstraction enable contextual flexibility during visual communication
Authors	Judith Fan, Robert Hawkins, Mike Wu, Noah Goodman
Abstract	Visual modes of communication are ubiquitous in modern life — from maps to data plots to political cartoons. Here we investigate drawing, the most basic form of visual communication. Participants were paired in an online environment to play a drawing-based reference game. On each trial, both participants were shown the same four objects, but in different locations. The sketcher’s goal was to draw one of these objects so that the viewer could select it from the array. On `close' trials, objects belonged to the same basic-level category, whereas on` far’ trials objects belonged to different categories. We found that people exploited shared information to efficiently communicate about the target object: on far trials, sketchers achieved high recognition accuracy while applying fewer strokes, using less ink, and spending less time on their drawings than on close trials. We hypothesized that humans succeed in this task by recruiting two core faculties: visual abstraction, the ability to perceive the correspondence between an object and a drawing of it; and pragmatic inference, the ability to judge what information would help a viewer distinguish the target from distractors. To evaluate this hypothesis, we developed a computational model of the sketcher that embodied both faculties, instantiated as a deep convolutional neural network nested within a probabilistic program. We found that this model fit human data well and outperformed lesioned variants. Together, this work provides the first algorithmically explicit theory of how visual perception and social cognition jointly support contextual flexibility in visual communication.
Tasks
Published	2019-03-11
URL	http://arxiv.org/abs/1903.04448v2
PDF	http://arxiv.org/pdf/1903.04448v2.pdf
PWC	https://paperswithcode.com/paper/pragmatic-inference-and-visual-abstraction
Repo
Framework

A neural network-based framework for financial model calibration


Title	A neural network-based framework for financial model calibration
Authors	Shuaiqiang Liu, Anastasia Borovykh, Lech A. Grzelak, Cornelis W. Oosterlee
Abstract	A data-driven approach called CaNN (Calibration Neural Network) is proposed to calibrate financial asset price models using an Artificial Neural Network (ANN). Determining optimal values of the model parameters is formulated as training hidden neurons within a machine learning framework, based on available financial option prices. The framework consists of two parts: a forward pass in which we train the weights of the ANN off-line, valuing options under many different asset model parameter settings; and a backward pass, in which we evaluate the trained ANN-solver on-line, aiming to find the weights of the neurons in the input layer. The rapid on-line learning of implied volatility by ANNs, in combination with the use of an adapted parallel global optimization method, tackles the computation bottleneck and provides a fast and reliable technique for calibrating model parameters while avoiding, as much as possible, getting stuck in local minima. Numerical experiments confirm that this machine-learning framework can be employed to calibrate parameters of high-dimensional stochastic volatility models efficiently and accurately.
Tasks	Calibration
Published	2019-04-23
URL	http://arxiv.org/abs/1904.10523v1
PDF	http://arxiv.org/pdf/1904.10523v1.pdf
PWC	https://paperswithcode.com/paper/a-neural-network-based-framework-for
Repo
Framework


Title	Twin-Finder: Integrated Reasoning Engine for Pointer-related Code Clone Detection
Authors	Hongfa Xue, Guru Venkataramani, Tian Lan
Abstract	Detecting code clones is crucial in various software engineering tasks. In particular, code clone detection can have significant uses in the context of analyzing and fixing bugs in large scale applications. However, prior works, such as machine learning based clone detection, may cause a considerable amount of false positives. In this paper, we propose Twin-Finder, a novel, closed-loop approach for pointer-related code clone detection that integrates machine learning and symbolic execution techniques to achieve precision. Twin-Finder introduces a clone verification mechanism to formally verify if two clone samples are indeed clones and a feedback loop to automatically generated formal rules to tune machine learning algorithm and further reduce the false positives. Our experimental results show Twin-Finder that can swiftly identify up 9X more code clones comparing to conventional code clone detection approaches. We conduct security analysis for memory safety using real-world applications Links version 2.14 and libreOffice-6.0.0.1. Twin-Finder is able to find 6 unreported bugs in Links version 2.14 and one public patched bug in libreOffice-6.0.0.1.
Tasks
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00561v2
PDF	https://arxiv.org/pdf/1911.00561v2.pdf
PWC	https://paperswithcode.com/paper/twin-finder-integrated-reasoning-engine-for
Repo
Framework

Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming


Title	Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming
Authors	Caio Corro, Ivan Titov
Abstract	We treat projective dependency trees as latent variables in our probabilistic model and induce them in such a way as to be beneficial for a downstream task, without relying on any direct tree supervision. Our approach relies on Gumbel perturbations and differentiable dynamic programming. Unlike previous approaches to latent tree learning, we stochastically sample global structures and our parser is fully differentiable. We illustrate its effectiveness on sentiment analysis and natural language inference tasks. We also study its properties on a synthetic structure induction task. Ablation studies emphasize the importance of both stochasticity and constraining latent structures to be projective trees.
Tasks	Natural Language Inference, Sentiment Analysis
Published	2019-06-24
URL	https://arxiv.org/abs/1906.09992v1
PDF	https://arxiv.org/pdf/1906.09992v1.pdf
PWC	https://paperswithcode.com/paper/learning-latent-trees-with-stochastic
Repo
Framework

A Review of Cooperative Multi-Agent Deep Reinforcement Learning


Title	A Review of Cooperative Multi-Agent Deep Reinforcement Learning
Authors	Afshin OroojlooyJadid, Davood Hajinezhad
Abstract	Deep Reinforcement Learning has made significant progress in multi-agent systems in recent years. In this review article, we have mostly focused on recent papers on Multi-Agent Reinforcement Learning (MARL) than the older papers, unless it was necessary. Several ideas and papers are proposed with different notations, and we tried our best to unify them with a single notation and categorize them by their relevance. In particular, we have focused on five common approaches on modeling and solving multi-agent reinforcement learning problems: (I) independent-learners, (II) fully observable critic, (III) value function decomposition, (IV) consensus, (IV) learn to communicate. Moreover, we discuss some new emerging research areas in MARL along with the relevant recent papers. In addition, some of the recent applications of MARL in real world are discussed. Finally, a list of available environments for MARL research are provided and the paper is concluded with proposals on the possible research directions.
Tasks	Multi-agent Reinforcement Learning
Published	2019-08-11
URL	https://arxiv.org/abs/1908.03963v2
PDF	https://arxiv.org/pdf/1908.03963v2.pdf
PWC	https://paperswithcode.com/paper/a-review-of-cooperative-multi-agent-deep
Repo
Framework

Better transfer learning with inferred successor maps


Title	Better transfer learning with inferred successor maps
Authors	Tamas J. Madarasz
Abstract	Humans and animals show remarkable flexibility in adjusting their behaviour when their goals, or rewards in the environment change. While such flexibility is a hallmark of intelligent behaviour, these multi-task scenarios remain an important challenge for machine learning algorithms and neurobiological models alike. We investigated two approaches that could enable this flexibility: factorized representations, which abstract away general aspects of a task from those prone to change, and nonparametric, memory-based approaches, which can provide a principled way of using similarity to past experiences to guide current behaviour. In particular, we combine the successor representation (SR) that factors the value of actions into expected outcomes and corresponding rewards with evaluating task similarity through clustering the space of reward functions. The proposed algorithm inverts a generative model over tasks, and dynamically samples from a flexible number of distinct SR maps while accumulating evidence about the current task context through amortized inference. It improves SR’s transfer capabilities and outperforms competing algorithms and baselines in settings with both known and unsignalled rewards changes. Further, as a neurobiological model of spatial coding in the hippocampus, it explains important signatures of this representation, such as the “flickering” behaviour of hippocampal maps, and trajectory-dependent place cells (so-called splitter cells) and their dynamics. We thus provide a novel algorithmic approach for multi-task learning, as well as a common normative framework that links together these different characteristics of the brain’s spatial representation.
Tasks	Multi-Task Learning, Transfer Learning
Published	2019-06-18
URL	https://arxiv.org/abs/1906.07663v5
PDF	https://arxiv.org/pdf/1906.07663v5.pdf
PWC	https://paperswithcode.com/paper/inferred-successor-maps-for-better-transfer
Repo
Framework

Inductive Logic Programming via Differentiable Deep Neural Logic Networks


Title	Inductive Logic Programming via Differentiable Deep Neural Logic Networks
Authors	Ali Payani, Faramarz Fekri
Abstract	We propose a novel paradigm for solving Inductive Logic Programming (ILP) problems via deep recurrent neural networks. This proposed ILP solver is designed based on differentiable implementation of the deduction via forward chaining. In contrast to the majority of past methods, instead of searching through the space of possible first-order logic rules by using some restrictive rule templates, we directly learn the symbolic logical predicate rules by introducing a novel differentiable Neural Logic (dNL) network. The proposed dNL network is able to learn and represent Boolean functions efficiently and in an explicit manner. We show that the proposed dNL-ILP solver supports desirable features such as recursion and predicate invention. Further, we investigate the performance of the proposed ILP solver in classification tasks involving benchmark relational datasets. In particular, we show that our proposed method outperforms the state of the art ILP solvers in classification tasks for Mutagenesis, Cora and IMDB datasets.
Tasks
Published	2019-06-08
URL	https://arxiv.org/abs/1906.03523v1
PDF	https://arxiv.org/pdf/1906.03523v1.pdf
PWC	https://paperswithcode.com/paper/inductive-logic-programming-via
Repo
Framework