October 19, 2019

2699 words 13 mins read

Paper Group ANR 134

Experimental Tests of Spirituality. Visualizing and Understanding Deep Neural Networks in CTR Prediction. On Variational Methods for Motion Compensated Inpainting. Provable Gaussian Embedding with One Observation. Confusion2Vec: Towards Enriching Vector Space Word Representations with Representational Ambiguities. VMAV-C: A Deep Attention-based Rei …

Experimental Tests of Spirituality


Title	Experimental Tests of Spirituality
Authors	Abraham Loeb
Abstract	We currently harness technologies that could shed new light on old philosophical questions, such as whether our mind entails anything beyond our body or whether our moral values reflect universal truth.
Tasks
Published	2018-06-04
URL	http://arxiv.org/abs/1806.01661v1
PDF	http://arxiv.org/pdf/1806.01661v1.pdf
PWC	https://paperswithcode.com/paper/experimental-tests-of-spirituality
Repo
Framework

Visualizing and Understanding Deep Neural Networks in CTR Prediction


Title	Visualizing and Understanding Deep Neural Networks in CTR Prediction
Authors	Lin Guo, Hui Ye, Wenbo Su, Henhuan Liu, Kai Sun, Hang Xiang
Abstract	Although deep learning techniques have been successfully applied to many tasks, interpreting deep neural network models is still a big challenge to us. Recently, many works have been done on visualizing and analyzing the mechanism of deep neural networks in the areas of image processing and natural language processing. In this paper, we present our approaches to visualize and understand deep neural networks for a very important commercial task–CTR (Click-through rate) prediction. We conduct experiments on the productive data from our online advertising system with daily varying distribution. To understand the mechanism and the performance of the model, we inspect the model’s inner status at neuron level. Also, a probe approach is implemented to measure the layer-wise performance of the model. Moreover, to measure the influence from the input features, we calculate saliency scores based on the back-propagated gradients. Practical applications are also discussed, for example, in understanding, monitoring, diagnosing and refining models and algorithms.
Tasks	Click-Through Rate Prediction
Published	2018-06-22
URL	http://arxiv.org/abs/1806.08541v1
PDF	http://arxiv.org/pdf/1806.08541v1.pdf
PWC	https://paperswithcode.com/paper/visualizing-and-understanding-deep-neural
Repo
Framework

On Variational Methods for Motion Compensated Inpainting


Title	On Variational Methods for Motion Compensated Inpainting
Authors	Francois Lauze, Mads Nielsen
Abstract	We develop in this paper a generic Bayesian framework for the joint estimation of motion and recovery of missing data in a damaged video sequence. Using standard maximum a posteriori to variational formulation rationale, we derive generic minimum energy formulations for the estimation of a reconstructed sequence as well as motion recovery. We instantiate these energy formulations and from their Euler-Lagrange Equations, we propose a full multiresolution algorithms in order to compute good local minimizers for our energies and discuss their numerical implementations, focusing on the missing data recovery part, i.e. inpainting. Experimental results for synthetic as well as real sequences are presented. Image sequences and extra material is available at http://image.diku.dk/francois/seqinp.php.
Tasks
Published	2018-09-21
URL	http://arxiv.org/abs/1809.07983v1
PDF	http://arxiv.org/pdf/1809.07983v1.pdf
PWC	https://paperswithcode.com/paper/on-variational-methods-for-motion-compensated
Repo
Framework

Provable Gaussian Embedding with One Observation


Title	Provable Gaussian Embedding with One Observation
Authors	Ming Yu, Zhuoran Yang, Tuo Zhao, Mladen Kolar, Zhaoran Wang
Abstract	The success of machine learning methods heavily relies on having an appropriate representation for data at hand. Traditionally, machine learning approaches relied on user-defined heuristics to extract features encoding structural information about data. However, recently there has been a surge in approaches that learn how to encode the data automatically in a low dimensional space. Exponential family embedding provides a probabilistic framework for learning low-dimensional representation for various types of high-dimensional data. Though successful in practice, theoretical underpinnings for exponential family embeddings have not been established. In this paper, we study the Gaussian embedding model and develop the first theoretical results for exponential family embedding models. First, we show that, under mild condition, the embedding structure can be learned from one observation by leveraging the parameter sharing between different contexts even though the data are dependent with each other. Second, we study properties of two algorithms used for learning the embedding structure and establish convergence results for each of them. The first algorithm is based on a convex relaxation, while the other solved the non-convex formulation of the problem directly. Experiments demonstrate the effectiveness of our approach.
Tasks
Published	2018-10-25
URL	http://arxiv.org/abs/1810.11098v1
PDF	http://arxiv.org/pdf/1810.11098v1.pdf
PWC	https://paperswithcode.com/paper/provable-gaussian-embedding-with-one
Repo
Framework

Confusion2Vec: Towards Enriching Vector Space Word Representations with Representational Ambiguities


Title	Confusion2Vec: Towards Enriching Vector Space Word Representations with Representational Ambiguities
Authors	Prashanth Gurunath Shivakumar, Panayiotis Georgiou
Abstract	Word vector representations are a crucial part of Natural Language Processing (NLP) and Human Computer Interaction. In this paper, we propose a novel word vector representation, Confusion2Vec, motivated from the human speech production and perception that encodes representational ambiguity. Humans employ both acoustic similarity cues and contextual cues to decode information and we focus on a model that incorporates both sources of information. The representational ambiguity of acoustics, which manifests itself in word confusions, is often resolved by both humans and machines through contextual cues. A range of representational ambiguities can emerge in various domains further to acoustic perception, such as morphological transformations, paraphrasing for NLP tasks like machine translation etc. In this work, we present a case study in application to Automatic Speech Recognition (ASR), where the word confusions are related to acoustic similarity. We present several techniques to train an acoustic perceptual similarity representation ambiguity. We term this Confusion2Vec and learn on unsupervised-generated data from ASR confusion networks or lattice-like structures. Appropriate evaluations for the Confusion2Vec are formulated for gauging acoustic similarity in addition to semantic-syntactic and word similarity evaluations. The Confusion2Vec is able to model word confusions efficiently, without compromising on the semantic-syntactic word relations, thus effectively enriching the word vector space with extra task relevant ambiguity information. We provide an intuitive exploration of the 2-dimensional Confusion2Vec space using Principal Component Analysis of the embedding and relate to semantic, syntactic and acoustic relationships. The potential of Confusion2Vec in the utilization of uncertainty present in lattices is demonstrated through small examples relating to ASR error correction.
Tasks	Machine Translation, Speech Recognition
Published	2018-11-08
URL	http://arxiv.org/abs/1811.03199v2
PDF	http://arxiv.org/pdf/1811.03199v2.pdf
PWC	https://paperswithcode.com/paper/confusion2vec-towards-enriching-vector-space
Repo
Framework

VMAV-C: A Deep Attention-based Reinforcement Learning Algorithm for Model-based Control


Title	VMAV-C: A Deep Attention-based Reinforcement Learning Algorithm for Model-based Control
Authors	Xingxing Liang, Qi Wang, Yanghe Feng, Zhong Liu, Jincai Huang
Abstract	Recent breakthroughs in Go play and strategic games have witnessed the great potential of reinforcement learning in intelligently scheduling in uncertain environment, but some bottlenecks are also encountered when we generalize this paradigm to universal complex tasks. Among them, the low efficiency of data utilization in model-free reinforcement algorithms is of great concern. In contrast, the model-based reinforcement learning algorithms can reveal underlying dynamics in learning environments and seldom suffer the data utilization problem. To address the problem, a model-based reinforcement learning algorithm with attention mechanism embedded is proposed as an extension of World Models in this paper. We learn the environment model through Mixture Density Network Recurrent Network(MDN-RNN) for agents to interact, with combinations of variational auto-encoder(VAE) and attention incorporated in state value estimates during the process of learning policy. In this way, agent can learn optimal policies through less interactions with actual environment, and final experiments demonstrate the effectiveness of our model in control problem.
Tasks	Deep Attention
Published	2018-12-24
URL	http://arxiv.org/abs/1812.09968v1
PDF	http://arxiv.org/pdf/1812.09968v1.pdf
PWC	https://paperswithcode.com/paper/vmav-c-a-deep-attention-based-reinforcement
Repo
Framework

Selection of Random Walkers that Optimizes the Global Mean First-Passage Time for Search in Complex Networks


Title	Selection of Random Walkers that Optimizes the Global Mean First-Passage Time for Search in Complex Networks
Authors	Mucong Ding, Kwok Yip Szeto
Abstract	We design a method to optimize the global mean first-passage time (GMFPT) of multiple random walkers searching in complex networks for a general target, without specifying the property of the target node. According to the Laplace transformed formula of the GMFPT, we can equivalently minimize the overlap between the probability distribution of sites visited by the random walkers. We employ a mutation only genetic algorithm to solve this optimization problem using a population of walkers with different starting positions and a corresponding mutation matrix to modify them. The numerical experiments on two kinds of random networks (WS and BA) show satisfactory results in selecting the origins for the walkers to achieve minimum overlap. Our method thus provides guidance for setting up the search process by multiple random walkers on complex networks.
Tasks
Published	2018-12-12
URL	http://arxiv.org/abs/1812.05058v1
PDF	http://arxiv.org/pdf/1812.05058v1.pdf
PWC	https://paperswithcode.com/paper/selection-of-random-walkers-that-optimizes
Repo
Framework

Robustness of classification ability of spiking neural networks


Title	Robustness of classification ability of spiking neural networks
Authors	Jie Yang, Pingping Zhang, Yan Liu
Abstract	It is well-known that the robustness of artificial neural networks (ANNs) is important for their wide ranges of applications. In this paper, we focus on the robustness of the classification ability of a spiking neural network which receives perturbed inputs. Actually, the perturbation is allowed to be arbitrary styles. However, Gaussian perturbation and other regular ones have been rarely investigated. For classification problems, the closer to the desired point, the more perturbed points there are in the input space. In addition, the perturbation may be periodic. Based on these facts, we only consider sinusoidal and Gaussian perturbations in this paper. With the SpikeProp algorithm, we perform extensive experiments on the classical XOR problem and other three benchmark datasets. The numerical results show that there is not significant reduction in the classification ability of the network if the input signals are subject to sinusoidal and Gaussian perturbations.
Tasks
Published	2018-01-30
URL	http://arxiv.org/abs/1801.09827v1
PDF	http://arxiv.org/pdf/1801.09827v1.pdf
PWC	https://paperswithcode.com/paper/robustness-of-classification-ability-of
Repo
Framework

Multi-Scale Distributed Representation for Deep Learning and its Application to b-Jet Tagging


Title	Multi-Scale Distributed Representation for Deep Learning and its Application to b-Jet Tagging
Authors	Jason Lee, Inkyu Park, Sangnam Park
Abstract	Recently machine learning algorithms based on deep layered artificial neural networks (DNNs) have been applied to a wide variety of high energy physics problems such as jet tagging or event classification. We explore a simple but effective preprocessing step which transforms each real-valued observational quantity or input feature into a binary number with a fixed number of digits. Each binary digit represents the quantity or magnitude in different scales. We have shown that this approach improves the performance of DNNs significantly for some specific tasks without any further complication in feature engineering. We apply this multi-scale distributed binary representation to deep learning on b-jet tagging using daughter particles’ momenta and vertex information.
Tasks	Feature Engineering
Published	2018-11-29
URL	http://arxiv.org/abs/1811.12069v1
PDF	http://arxiv.org/pdf/1811.12069v1.pdf
PWC	https://paperswithcode.com/paper/multi-scale-distributed-representation-for
Repo
Framework

Towards Learning Transferable Conversational Skills using Multi-dimensional Dialogue Modelling


Title	Towards Learning Transferable Conversational Skills using Multi-dimensional Dialogue Modelling
Authors	Simon Keizer, Verena Rieser
Abstract	Recent statistical approaches have improved the robustness and scalability of spoken dialogue systems. However, despite recent progress in domain adaptation, their reliance on in-domain data still limits their cross-domain scalability. In this paper, we argue that this problem can be addressed by extending current models to reflect and exploit the multi-dimensional nature of human dialogue. We present our multi-dimensional, statistical dialogue management framework, in which transferable conversational skills can be learnt by separating out domain-independent dimensions of communication and using multi-agent reinforcement learning. Our initial experiments with a simulated user show that we can speed up the learning process by transferring learnt policies.
Tasks	Dialogue Management, Domain Adaptation, Multi-agent Reinforcement Learning, Spoken Dialogue Systems
Published	2018-03-31
URL	http://arxiv.org/abs/1804.00146v1
PDF	http://arxiv.org/pdf/1804.00146v1.pdf
PWC	https://paperswithcode.com/paper/towards-learning-transferable-conversational
Repo
Framework

Patient Risk Assessment and Warning Symptom Detection Using Deep Attention-Based Neural Networks


Title	Patient Risk Assessment and Warning Symptom Detection Using Deep Attention-Based Neural Networks
Authors	Ivan Girardi, Pengfei Ji, An-phi Nguyen, Nora Hollenstein, Adam Ivankay, Lorenz Kuhn, Chiara Marchiori, Ce Zhang
Abstract	We present an operational component of a real-world patient triage system. Given a specific patient presentation, the system is able to assess the level of medical urgency and issue the most appropriate recommendation in terms of best point of care and time to treat. We use an attention-based convolutional neural network architecture trained on 600,000 doctor notes in German. We compare two approaches, one that uses the full text of the medical notes and one that uses only a selected list of medical entities extracted from the text. These approaches achieve 79% and 66% precision, respectively, but on a confidence threshold of 0.6, precision increases to 85% and 75%, respectively. In addition, a method to detect warning symptoms is implemented to render the classification task transparent from a medical perspective. The method is based on the learning of attention scores and a method of automatic validation using the same data.
Tasks	Deep Attention
Published	2018-09-28
URL	http://arxiv.org/abs/1809.10804v1
PDF	http://arxiv.org/pdf/1809.10804v1.pdf
PWC	https://paperswithcode.com/paper/patient-risk-assessment-and-warning-symptom
Repo
Framework

Stochastic modified equations for the asynchronous stochastic gradient descent


Title	Stochastic modified equations for the asynchronous stochastic gradient descent
Authors	Jing An, Jianfeng Lu, Lexing Ying
Abstract	We propose a stochastic modified equations (SME) for modeling the asynchronous stochastic gradient descent (ASGD) algorithms. The resulting SME of Langevin type extracts more information about the ASGD dynamics and elucidates the relationship between different types of stochastic gradient algorithms. We show the convergence of ASGD to the SME in the continuous time limit, as well as the SME’s precise prediction to the trajectories of ASGD with various forcing terms. As an application of the SME, we propose an optimal mini-batching strategy for ASGD via solving the optimal control problem of the associated SME.
Tasks
Published	2018-05-21
URL	https://arxiv.org/abs/1805.08244v3
PDF	https://arxiv.org/pdf/1805.08244v3.pdf
PWC	https://paperswithcode.com/paper/stochastic-modified-equations-for-the
Repo
Framework

Deep attention-guided fusion network for lesion segmentation


Title	Deep attention-guided fusion network for lesion segmentation
Authors	Hengliang Zhu, Yangyang Hao, Lizhuang Ma, Ruixing Li, Hua Wang
Abstract	We participated the Task 1: Lesion Segmentation. The paper describes our algorithm and the final result of validation set for the ISIC Challenge 2018 - Skin Lesion Analysis Towards Melanoma Detection.
Tasks	Deep Attention, Lesion Segmentation
Published	2018-07-23
URL	http://arxiv.org/abs/1807.08471v2
PDF	http://arxiv.org/pdf/1807.08471v2.pdf
PWC	https://paperswithcode.com/paper/deep-attention-guided-fusion-network-for
Repo
Framework

Automatic Target Recovery for Hindi-English Code Mixed Puns


Title	Automatic Target Recovery for Hindi-English Code Mixed Puns
Authors	Srishti Aggarwal, Kritik Mathur, Radhika Mamidi
Abstract	In order for our computer systems to be more human-like, with a higher emotional quotient, they need to be able to process and understand intrinsic human language phenomena like humour. In this paper, we consider a subtype of humour - puns, which are a common type of wordplay-based jokes. In particular, we consider code-mixed puns which have become increasingly mainstream on social media, in informal conversations and advertisements and aim to build a system which can automatically identify the pun location and recover the target of such puns. We first study and classify code-mixed puns into two categories namely intra-sentential and intra-word, and then propose a four-step algorithm to recover the pun targets for puns belonging to the intra-sentential category. Our algorithm uses language models, and phonetic similarity-based features to get the desired results. We test our approach on a small set of code-mixed punning advertisements, and observe that our system is successfully able to recover the targets for 67% of the puns.
Tasks
Published	2018-06-11
URL	http://arxiv.org/abs/1806.04535v1
PDF	http://arxiv.org/pdf/1806.04535v1.pdf
PWC	https://paperswithcode.com/paper/automatic-target-recovery-for-hindi-english
Repo
Framework

Compact Optimization Algorithms with Re-sampled Inheritance


Title	Compact Optimization Algorithms with Re-sampled Inheritance
Authors	Giovanni Iacca, Fabio Caraffini
Abstract	Compact optimization algorithms are a class of Estimation of Distribution Algorithms (EDAs) characterized by extremely limited memory requirements (hence they are called “compact”). As all EDAs, compact algorithms build and update a probabilistic model of the distribution of solutions within the search space, as opposed to population-based algorithms that instead make use of an explicit population of solutions. In addition to that, to keep their memory consumption low, compact algorithms purposely employ simple probabilistic models that can be described with a small number of parameters. Despite their simplicity, compact algorithms have shown good performances on a broad range of benchmark functions and real-world problems. However, compact algorithms also come with some drawbacks, i.e. they tend to premature convergence and show poorer performance on non-separable problems. To overcome these limitations, here we investigate a possible algorithmic scheme obtained by combining compact algorithms with a non-disruptive restart mechanism taken from the literature, named Re-Sampled Inheritance (RI). The resulting compact algorithms with RI are tested on the CEC 2014 benchmark functions. The numerical results show on the one hand that the use of RI consistently enhances the performances of compact algorithms, still keeping a limited usage of memory. On the other hand, our experiments show that among the tested algorithms, the best performance is obtained by compact Differential Evolution with RI.
Tasks
Published	2018-09-12
URL	http://arxiv.org/abs/1809.04343v3
PDF	http://arxiv.org/pdf/1809.04343v3.pdf
PWC	https://paperswithcode.com/paper/compact-optimization-algorithms-with-re
Repo
Framework