April 2, 2020

3155 words 15 mins read

Paper Group ANR 161

Conditional Self-Attention for Query-based Summarization. Data-Driven Factor Graphs for Deep Symbol Detection. Composition of kernel and acquisition functions for High Dimensional Bayesian Optimization. A physics-informed feature engineering approach to use machine learning with limited amounts of data for alloy design: shape memory alloy demonstra …

Conditional Self-Attention for Query-based Summarization


Title	Conditional Self-Attention for Query-based Summarization
Authors	Yujia Xie, Tianyi Zhou, Yi Mao, Weizhu Chen
Abstract	Self-attention mechanisms have achieved great success on a variety of NLP tasks due to its flexibility of capturing dependency between arbitrary positions in a sequence. For problems such as query-based summarization (Qsumm) and knowledge graph reasoning where each input sequence is associated with an extra query, explicitly modeling such conditional contextual dependencies can lead to a more accurate solution, which however cannot be captured by existing self-attention mechanisms. In this paper, we propose \textit{conditional self-attention} (CSA), a neural network module designed for conditional dependency modeling. CSA works by adjusting the pairwise attention between input tokens in a self-attention module with the matching score of the inputs to the given query. Thereby, the contextual dependencies modeled by CSA will be highly relevant to the query. We further studied variants of CSA defined by different types of attention. Experiments on Debatepedia and HotpotQA benchmark datasets show CSA consistently outperforms vanilla Transformer and previous models for the Qsumm problem.
Tasks
Published	2020-02-18
URL	https://arxiv.org/abs/2002.07338v1
PDF	https://arxiv.org/pdf/2002.07338v1.pdf
PWC	https://paperswithcode.com/paper/conditional-self-attention-for-query-based
Repo
Framework

Data-Driven Factor Graphs for Deep Symbol Detection


Title	Data-Driven Factor Graphs for Deep Symbol Detection
Authors	Nir Shlezinger, Nariman Farsad, Yonina C. Eldar, Andrea J. Goldsmith
Abstract	Many important schemes in signal processing and communications, ranging from the BCJR algorithm to the Kalman filter, are instances of factor graph methods. This family of algorithms is based on recursive message passing-based computations carried out over graphical models, representing a factorization of the underlying statistics. Consequently, in order to implement these algorithms, one must have accurate knowledge of the statistical model of the considered signals. In this work we propose to implement factor graph methods in a data-driven manner. In particular, we propose to use machine learning (ML) tools to learn the factor graph, instead of the overall system task, which in turn is used for inference by message passing over the learned graph. We apply the proposed approach to learn the factor graph representing a finite-memory channel, demonstrating the resulting ability to implement BCJR detection in a data-driven fashion. We demonstrate that the proposed system, referred to as BCJRNet, learns to implement the BCJR algorithm from a small training set, and that the resulting receiver exhibits improved robustness to inaccurate training compared to the conventional channel-model-based receiver operating under the same level of uncertainty. Our results indicate that by utilizing ML tools to learn factor graphs from labeled data, one can implement a broad range of model-based algorithms, which traditionally require full knowledge of the underlying statistics, in a data-driven fashion.
Tasks
Published	2020-01-31
URL	https://arxiv.org/abs/2002.00758v1
PDF	https://arxiv.org/pdf/2002.00758v1.pdf
PWC	https://paperswithcode.com/paper/data-driven-factor-graphs-for-deep-symbol
Repo
Framework

Composition of kernel and acquisition functions for High Dimensional Bayesian Optimization


Title	Composition of kernel and acquisition functions for High Dimensional Bayesian Optimization
Authors	Antonio Candelieri, Ilaria Giordani, Riccardo Perego, Francesco Archetti
Abstract	Bayesian Optimization has become the reference method for the global optimization of black box, expensive and possibly noisy functions. Bayesian Op-timization learns a probabilistic model about the objective function, usually a Gaussian Process, and builds, depending on its mean and variance, an acquisition function whose optimizer yields the new evaluation point, leading to update the probabilistic surrogate model. Despite its sample efficiency, Bayesian Optimiza-tion does not scale well with the dimensions of the problem. The optimization of the acquisition function has received less attention because its computational cost is usually considered negligible compared to that of the evaluation of the objec-tive function. Its efficient optimization is often inhibited, particularly in high di-mensional problems, by multiple extrema. In this paper we leverage the addition-ality of the objective function into mapping both the kernel and the acquisition function of the Bayesian Optimization in lower dimensional subspaces. This ap-proach makes more efficient the learning/updating of the probabilistic surrogate model and allows an efficient optimization of the acquisition function. Experi-mental results are presented for real-life application, that is the control of pumps in urban water distribution systems.
Tasks
Published	2020-03-09
URL	https://arxiv.org/abs/2003.04207v1
PDF	https://arxiv.org/pdf/2003.04207v1.pdf
PWC	https://paperswithcode.com/paper/composition-of-kernel-and-acquisition
Repo
Framework

A physics-informed feature engineering approach to use machine learning with limited amounts of data for alloy design: shape memory alloy demonstration


Title	A physics-informed feature engineering approach to use machine learning with limited amounts of data for alloy design: shape memory alloy demonstration
Authors	Sen Liu, Branden B. Kappes, Behnam Amin-ahmadi, Othmane Benafan, Aaron P. Stebner, Xiaoli Zhang
Abstract	Machine learning using limited data from physical experiments is shown to work to predict new shape memory alloys in a high dimensional alloy design space that considers chemistry and thermal post-processing. The key to enabling the machine learning algorithms to make predictions of new alloys and their post-processing is shown to be a physics-informed featured engineering approach. Specifically, elemental features previously engineered by the computational materials community to model composition effects in materials are combined with newly engineered heat treatment features. These new features result from pre-processing the heat treatment data using mathematical relationships known to describe the thermodynamics and kinetics of precipitation in alloys. The prior application of the nonlinear physical models to the data in effect linearizes some of the complex alloy development trends a priori using known physics, and results in greatly improved performance of the ML algorithms trained on relatively few data points.
Tasks	Feature Engineering
Published	2020-03-04
URL	https://arxiv.org/abs/2003.01878v1
PDF	https://arxiv.org/pdf/2003.01878v1.pdf
PWC	https://paperswithcode.com/paper/a-physics-informed-feature-engineering
Repo
Framework

Two-Step Surface Damage Detection Scheme using Convolutional Neural Network and Artificial Neural Neural


Title	Two-Step Surface Damage Detection Scheme using Convolutional Neural Network and Artificial Neural Neural
Authors	Alice Yi Yang, Ling Cheng
Abstract	Surface damage on concrete is important as the damage can affect the structural integrity of the structure. This paper proposes a two-step surface damage detection scheme using Convolutional Neural Network (CNN) and Artificial Neural Network (ANN). The CNN classifies given input images into two categories: positive and negative. The positive category is where the surface damage is present within the image, otherwise the image is classified as negative. This is an image-based classification. The ANN accepts image inputs that have been classified as positive by the ANN. This reduces the number of images that are further processed by the ANN. The ANN performs feature-based classification, in which the features are extracted from the detected edges within the image. The edges are detected using Canny edge detection. A total of 19 features are extracted from the detected edges. These features are inputs into the ANN. The purpose of the ANN is to highlight only the positive damaged edges within the image. The CNN achieves an accuracy of 80.7% for image classification and the ANN achieves an accuracy of 98.1% for surface detection. The decreased accuracy in the CNN is due to the false positive detection, however false positives are tolerated whereas false negatives are not. The false negative detection for both CNN and ANN in the two-step scheme are 0%.
Tasks	Edge Detection, Image Classification
Published	2020-03-24
URL	https://arxiv.org/abs/2003.10760v1
PDF	https://arxiv.org/pdf/2003.10760v1.pdf
PWC	https://paperswithcode.com/paper/two-step-surface-damage-detection-scheme
Repo
Framework

Assessing Image Quality Issues for Real-World Problems


Title	Assessing Image Quality Issues for Real-World Problems
Authors	Tai-Yin Chiu, Yinan Zhao, Danna Gurari
Abstract	We introduce a new large-scale dataset that links the assessment of image quality issues to two practical vision tasks: image captioning and visual question answering. First, we identify for 39,181 images taken by people who are blind whether each is sufficient quality to recognize the content as well as what quality flaws are observed from six options. These labels serve as a critical foundation for us to make the following contributions: (1) a new problem and algorithms for deciding whether an image is insufficient quality to recognize the content and so not captionable, (2) a new problem and algorithms for deciding which of six quality flaws an image contains, (3) a new problem and algorithms for deciding whether a visual question is unanswerable due to unrecognizable content versus the content of interest being missing from the field of view, and (4) a novel application of more efficiently creating a large-scale image captioning dataset by automatically deciding whether an image is insufficient quality and so should not be captioned. We publicly-share our datasets and code to facilitate future extensions of this work: https://vizwiz.org.
Tasks	Image Captioning, Question Answering, Visual Question Answering
Published	2020-03-27
URL	https://arxiv.org/abs/2003.12511v2
PDF	https://arxiv.org/pdf/2003.12511v2.pdf
PWC	https://paperswithcode.com/paper/assessing-image-quality-issues-for-real-world
Repo
Framework

Development of an Expert System for Diabetic Type-2 Diet


Title	Development of an Expert System for Diabetic Type-2 Diet
Authors	Ibrahim M. Ahmed, Abeer M. Mahmoud
Abstract	A successful intelligent control of patient food for treatment purpose must combines patient interesting food list and doctors efficient treatment food list. Actually, many rural communities in Sudan have extremely limited access to diabetic diet centers. People travel long distances to clinics or medical facilities, and there is a shortage of medical experts in most of these facilities. This results in slow service, and patients end up waiting long hours without receiving any attention. Hence diabetic diet expert systems can play a significant role in such cases where medical experts are not readily available. This paper presents the design and implementation of an intelligent medical expert system for diabetes diet that intended to be used in Sudan. The development of the proposed expert system went through a number of stages such problem and need identification, requirements analysis, knowledge acquisition, formalization, design and implementation. Visual prolog was used for designing the graphical user interface and the implementation of the system. The proposed expert system is a promising helpful tool that reduces the workload for physicians and provides diabetics with simple and valuable assistance.
Tasks
Published	2020-02-22
URL	https://arxiv.org/abs/2003.05104v1
PDF	https://arxiv.org/pdf/2003.05104v1.pdf
PWC	https://paperswithcode.com/paper/development-of-an-expert-system-for-diabetic
Repo
Framework

Deep Image Translation with an Affinity-Based Change Prior for Unsupervised Multimodal Change Detection


Title	Deep Image Translation with an Affinity-Based Change Prior for Unsupervised Multimodal Change Detection
Authors	Luigi Tommaso Luppino, Michael Kampffmeyer, Filippo Maria Bianchi, Gabriele Moser, Sebastiano Bruno Serpico, Robert Jenssen, Stian Normann Anfinsen
Abstract	Image translation with convolutional neural networks has recently been used as an approach to multimodal change detection. Existing approaches train the networks by exploiting supervised information of the change areas, which, however, is not always available. A main challenge in the unsupervised problem setting is to avoid that change pixels affect the learning of the translation function. We propose two new network architectures trained with loss functions weighted by priors that reduce the impact of change pixels on the learning objective. The change prior is derived in an unsupervised fashion from relational pixel information captured by domain-specific affinity matrices. Specifically, we use the vertex degrees associated with an absolute affinity difference matrix and demonstrate their utility in combination with cycle consistency and adversarial training. The proposed neural networks are compared with state-of-the-art algorithms. Experiments conducted on two real datasets show the effectiveness of our methodology.
Tasks
Published	2020-01-13
URL	https://arxiv.org/abs/2001.04271v1
PDF	https://arxiv.org/pdf/2001.04271v1.pdf
PWC	https://paperswithcode.com/paper/deep-image-translation-with-an-affinity-based
Repo
Framework

Advanced Deep Learning Methodologies for Skin Cancer Classification in Prodromal Stages


Title	Advanced Deep Learning Methodologies for Skin Cancer Classification in Prodromal Stages
Authors	Muhammad Ali Farooq, Asma Khatoon, Viktor Varkarakis, Peter Corcoran
Abstract	Technology-assisted platforms provide reliable solutions in almost every field these days. One such important application in the medical field is the skin cancer classification in preliminary stages that need sensitive and precise data analysis. For the proposed study the Kaggle skin cancer dataset is utilized. The proposed study consists of two main phases. In the first phase, the images are preprocessed to remove the clutters thus producing a refined version of training images. To achieve that, a sharpening filter is applied followed by a hair removal algorithm. Different image quality measurement metrics including Peak Signal to Noise (PSNR), Mean Square Error (MSE), Maximum Absolute Squared Deviation (MXERR) and Energy Ratio/ Ratio of Squared Norms (L2RAT) are used to compare the overall image quality before and after applying preprocessing operations. The results from the aforementioned image quality metrics prove that image quality is not compromised however it is upgraded by applying the preprocessing operations. The second phase of the proposed research work incorporates deep learning methodologies that play an imperative role in accurate, precise and robust classification of the lesion mole. This has been reflected by using two state of the art deep learning models: Inception-v3 and MobileNet. The experimental results demonstrate notable improvement in train and validation accuracy by using the refined version of images of both the networks, however, the Inception-v3 network was able to achieve better validation accuracy thus it was finally selected to evaluate it on test data. The final test accuracy using state of art Inception-v3 network was 86%.
Tasks	Skin Cancer Classification
Published	2020-03-13
URL	https://arxiv.org/abs/2003.06356v1
PDF	https://arxiv.org/pdf/2003.06356v1.pdf
PWC	https://paperswithcode.com/paper/advanced-deep-learning-methodologies-for-skin
Repo
Framework

Evolutionary Optimization of Deep Learning Activation Functions


Title	Evolutionary Optimization of Deep Learning Activation Functions
Authors	Garrett Bingham, William Macke, Risto Miikkulainen
Abstract	The choice of activation function can have a large effect on the performance of a neural network. While there have been some attempts to hand-engineer novel activation functions, the Rectified Linear Unit (ReLU) remains the most commonly-used in practice. This paper shows that evolutionary algorithms can discover novel activation functions that outperform ReLU. A tree-based search space of candidate activation functions is defined and explored with mutation, crossover, and exhaustive search. Experiments on training wide residual networks on the CIFAR-10 and CIFAR-100 image datasets show that this approach is effective. Replacing ReLU with evolved activation functions results in statistically significant increases in network accuracy. Optimal performance is achieved when evolution is allowed to customize activation functions to a particular task; however, these novel activation functions are shown to generalize, achieving high performance across tasks. Evolutionary optimization of activation functions is therefore a promising new dimension of metalearning in neural networks.
Tasks
Published	2020-02-17
URL	https://arxiv.org/abs/2002.07224v1
PDF	https://arxiv.org/pdf/2002.07224v1.pdf
PWC	https://paperswithcode.com/paper/evolutionary-optimization-of-deep-learning
Repo
Framework

SCATTER: Selective Context Attentional Scene Text Recognizer


Title	SCATTER: Selective Context Attentional Scene Text Recognizer
Authors	Ron Litman, Oron Anschel, Shahar Tsiper, Roee Litman, Shai Mazor, R. Manmatha
Abstract	Scene Text Recognition (STR), the task of recognizing text against complex image backgrounds, is an active area of research. Current state-of-the-art (SOTA) methods still struggle to recognize text written in arbitrary shapes. In this paper, we introduce a novel architecture for STR, named Selective Context ATtentional Text Recognizer (SCATTER). SCATTER utilizes a stacked block architecture with intermediate supervision during training, that paves the way to successfully train a deep BiLSTM encoder, thus improving the encoding of contextual dependencies. Decoding is done using a two-step 1D attention mechanism. The first attention step re-weights visual features from a CNN backbone together with contextual features computed by a BiLSTM layer. The second attention step, similar to previous papers, treats the features as a sequence and attends to the intra-sequence relationships. Experiments show that the proposed approach surpasses SOTA performance on irregular text recognition benchmarks by 3.7% on average.
Tasks	Irregular Text Recognition, Scene Text Recognition
Published	2020-03-25
URL	https://arxiv.org/abs/2003.11288v1
PDF	https://arxiv.org/pdf/2003.11288v1.pdf
PWC	https://paperswithcode.com/paper/scatter-selective-context-attentional-scene
Repo
Framework

LocoGAN – Locally Convolutional GAN


Title	LocoGAN – Locally Convolutional GAN
Authors	Łukasz Struski, Szymon Knop, Jacek Tabor, Wiktor Daniec, Przemysław Spurek
Abstract	In the paper we construct a fully convolutional GAN model: LocoGAN, which latent space is given by noise-like images of possibly different resolutions. The learning is local, i.e. we process not the whole noise-like image, but the sub-images of a fixed size. As a consequence LocoGAN can produce images of arbitrary dimensions e.g. LSUN bedroom data set. Another advantage of our approach comes from the fact that we use the position channels, which allows the generation of fully periodic (e.g. cylindrical panoramic images) or almost periodic ,,infinitely long” images (e.g. wall-papers).
Tasks
Published	2020-02-18
URL	https://arxiv.org/abs/2002.07897v1
PDF	https://arxiv.org/pdf/2002.07897v1.pdf
PWC	https://paperswithcode.com/paper/locogan-locally-convolutional-gan
Repo
Framework

A New Perspective for Flexible Feature Gathering in Scene Text Recognition Via Character Anchor Pooling


Title	A New Perspective for Flexible Feature Gathering in Scene Text Recognition Via Character Anchor Pooling
Authors	Shangbang Long, Yushuo Guan, Kaigui Bian, Cong Yao
Abstract	Irregular scene text recognition has attracted much attention from the research community, mainly due to the complexity of shapes of text in natural scene. However, recent methods either rely on shape-sensitive modules such as bounding box regression, or discard sequence learning. To tackle these issues, we propose a pair of coupling modules, termed as Character Anchoring Module (CAM) and Anchor Pooling Module (APM), to extract high-level semantics from two-dimensional space to form feature sequences. The proposed CAM localizes the text in a shape-insensitive way by design by anchoring characters individually. APM then interpolates and gathers features flexibly along the character anchors which enables sequence learning. The complementary modules realize a harmonic unification of spatial information and sequence learning. With the proposed modules, our recognition system surpasses previous state-of-the-art scores on irregular and perspective text datasets, including, ICDAR 2015, CUTE, and Total-Text, while paralleling state-of-the-art performance on regular text datasets.
Tasks	Scene Text Recognition
Published	2020-02-10
URL	https://arxiv.org/abs/2002.03509v1
PDF	https://arxiv.org/pdf/2002.03509v1.pdf
PWC	https://paperswithcode.com/paper/a-new-perspective-for-flexible-feature
Repo
Framework

The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence


Title	The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
Authors	Gary Marcus
Abstract	Recent research in artificial intelligence and machine learning has largely emphasized general-purpose learning and ever-larger training sets and more and more compute. In contrast, I propose a hybrid, knowledge-driven, reasoning-based approach, centered around cognitive models, that could provide the substrate for a richer, more robust AI than is currently possible.
Tasks
Published	2020-02-14
URL	https://arxiv.org/abs/2002.06177v3
PDF	https://arxiv.org/pdf/2002.06177v3.pdf
PWC	https://paperswithcode.com/paper/the-next-decade-in-ai-four-steps-towards
Repo
Framework

Causal Inference under Networked Interference


Title	Causal Inference under Networked Interference
Authors	Yunpu Ma, Yuyi Wang, Volker Tresp
Abstract	Estimating individual treatment effects from data of randomized experiments is a critical task in causal inference. The Stable Unit Treatment Value Assumption (SUTVA) is usually made in causal inference. However, interference can introduce bias when the assigned treatment on one unit affects the potential outcomes of the neighboring units. This interference phenomenon is known as spillover effect in economics or peer effect in social science. Usually, in randomized experiments or observational studies with interconnected units, one can only observe treatment responses under interference. Hence, how to estimate the superimposed causal effect and recover the individual treatment effect in the presence of interference becomes a challenging task in causal inference. In this work, we study causal effect estimation under general network interference using GNNs, which are powerful tools for capturing the dependency in the graph. After deriving causal effect estimators, we further study intervention policy improvement on the graph under capacity constraint. We give policy regret bounds under network interference and treatment capacity constraint. Furthermore, a heuristic graph structure-dependent error bound for GNN-based causal estimators is provided.
Tasks	Causal Inference
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08506v1
PDF	https://arxiv.org/pdf/2002.08506v1.pdf
PWC	https://paperswithcode.com/paper/causal-inference-under-networked-interference
Repo
Framework