January 29, 2020

3203 words 16 mins read

Paper Group ANR 607

Of Cores: A Partial-Exploration Framework for Markov Decision Processes. Solving Visual Object Ambiguities when Pointing: An Unsupervised Learning Approach. Attenuating Random Noise in Seismic Data by a Deep Learning Approach. Machine Learning Kernel Method from a Quantum Generative Model. Sequence-to-Sequence Natural Language to Humanoid Robot Sig …

Of Cores: A Partial-Exploration Framework for Markov Decision Processes


Title	Of Cores: A Partial-Exploration Framework for Markov Decision Processes
Authors	Jan Křetínský, Tobias Meggendorfer
Abstract	We introduce a framework for approximate analysis of Markov decision processes (MDP) with bounded-, unbounded-, and infinite-horizon properties. The main idea is to identify a “core” of an MDP, i.e., a subsystem where we provably remain with high probability, and to avoid computation on the less relevant rest of the state space. Although we identify the core using simulations and statistical techniques, it allows for rigorous error bounds in the analysis. Consequently, we obtain efficient analysis algorithms based on partial exploration for various settings, including the challenging case of strongly connected systems.
Tasks
Published	2019-06-17
URL	https://arxiv.org/abs/1906.06931v2
PDF	https://arxiv.org/pdf/1906.06931v2.pdf
PWC	https://paperswithcode.com/paper/of-cores-a-partial-exploration-framework-for
Repo
Framework

Solving Visual Object Ambiguities when Pointing: An Unsupervised Learning Approach


Title	Solving Visual Object Ambiguities when Pointing: An Unsupervised Learning Approach
Authors	Doreen Jirak, David Biertimpel, Matthias Kerzel, Stefan Wermter
Abstract	Whenever we are addressing a specific object or refer to a certain spatial location, we are using referential or deictic gestures usually accompanied by some verbal description. Especially pointing gestures are necessary to dissolve ambiguities in a scene and they are of crucial importance when verbal communication may fail due to environmental conditions or when two persons simply do not speak the same language. With the currently increasing advances of humanoid robots and their future integration in domestic domains, the development of gesture interfaces complementing human-robot interaction scenarios is of substantial interest. The implementation of an intuitive gesture scenario is still challenging because both the pointing intention and the corresponding object have to be correctly recognized in real-time. The demand increases when considering pointing gestures in a cluttered environment, as is the case in households. Also, humans perform pointing in many different ways and those variations have to be captured. Research in this field often proposes a set of geometrical computations which do not scale well with the number of gestures and objects, use specific markers or a predefined set of pointing directions. In this paper, we propose an unsupervised learning approach to model the distribution of pointing gestures using a growing-when-required (GWR) network. We introduce an interaction scenario with a humanoid robot and define so-called ambiguity classes. Our implementation for the hand and object detection is independent of any markers or skeleton models, thus it can be easily reproduced. Our evaluation comparing a baseline computer vision approach with our GWR model shows that the pointing-object association is well learned even in cases of ambiguities resulting from close object proximity.
Tasks	Object Detection
Published	2019-12-13
URL	https://arxiv.org/abs/1912.06449v1
PDF	https://arxiv.org/pdf/1912.06449v1.pdf
PWC	https://paperswithcode.com/paper/solving-visual-object-ambiguities-when
Repo
Framework

Attenuating Random Noise in Seismic Data by a Deep Learning Approach


Title	Attenuating Random Noise in Seismic Data by a Deep Learning Approach
Authors	Xing Zhao, Ping Lu, Yanyan Zhang, Jianxiong Chen, Xiaoyang Li
Abstract	In the geophysical field, seismic noise attenuation has been considered as a critical and long-standing problem, especially for the pre-stack data processing. Here, we propose a model to leverage the deep-learning model for this task. Rather than directly applying an existing de-noising model from ordinary images to the seismic data, we have designed a particular deep-learning model, based on residual neural networks. It is named as N2N-Seismic, which has a strong ability to recover the seismic signals back to intact condition with the preservation of primary signals. The proposed model, achieving with great success in attenuating noise, has been tested on two different seismic datasets. Several metrics show that our method outperforms conventional approaches in terms of Signal-to-Noise-Ratio, Mean-Squared-Error, Phase Spectrum, etc. Moreover, robust tests in terms of effectively removing random noise from any dataset with strong and weak noises have been extensively scrutinized in making sure that the proposed model is able to maintain a good level of adaptation while dealing with large variations of noise characteristics and intensities.
Tasks
Published	2019-10-28
URL	https://arxiv.org/abs/1910.12800v1
PDF	https://arxiv.org/pdf/1910.12800v1.pdf
PWC	https://paperswithcode.com/paper/attenuating-random-noise-in-seismic-data-by-a
Repo
Framework

Machine Learning Kernel Method from a Quantum Generative Model


Title	Machine Learning Kernel Method from a Quantum Generative Model
Authors	Przemysław Sadowski
Abstract	Recently the use of Noisy Intermediate Scale Quantum (NISQ) devices for machine learning tasks has been proposed. The propositions often perform poorly due to various restrictions. However, the quantum devices should perform well in sampling tasks. Thus, we recall theory of sampling-based approach to machine learning and propose a quantum sampling based classifier. Namely, we use randomized feature map approach. We propose a method of quantum sampling based on random quantum circuits with parametrized rotations distribution. We obtain simple to use method with intuitive hyper-parameters that performs at least equally well as top out-of-the-box classical methods. In short we obtain a competitive quantum classifier with crucial component being quantum sampling – a promising task for quantum supremacy.
Tasks
Published	2019-07-11
URL	https://arxiv.org/abs/1907.05103v1
PDF	https://arxiv.org/pdf/1907.05103v1.pdf
PWC	https://paperswithcode.com/paper/machine-learning-kernel-method-from-a-quantum
Repo
Framework

Sequence-to-Sequence Natural Language to Humanoid Robot Sign Language


Title	Sequence-to-Sequence Natural Language to Humanoid Robot Sign Language
Authors	Jennifer J. Gago, Valentina Vasco, Bartek Łukawski, Ugo Pattacini, Vadim Tikhanoff, Juan G. Victores, Carlos Balaguer
Abstract	This paper presents a study on natural language to sign language translation with human-robot interaction application purposes. By means of the presented methodology, the humanoid robot TEO is expected to represent Spanish sign language automatically by converting text into movements, thanks to the performance of neural networks. Natural language to sign language translation presents several challenges to developers, such as the discordance between the length of input and output data and the use of non-manual markers. Therefore, neural networks and, consequently, sequence-to-sequence models, are selected as a data-driven system to avoid traditional expert system approaches or temporal dependencies limitations that lead to limited or too complex translation systems. To achieve these objectives, it is necessary to find a way to perform human skeleton acquisition in order to collect the signing input data. OpenPose and skeletonRetriever are proposed for this purpose and a 3D sensor specification study is developed to select the best acquisition hardware.
Tasks	Sign Language Translation
Published	2019-07-09
URL	https://arxiv.org/abs/1907.04198v1
PDF	https://arxiv.org/pdf/1907.04198v1.pdf
PWC	https://paperswithcode.com/paper/sequence-to-sequence-natural-language-to
Repo
Framework

Low-dimensional Embodied Semantics for Music and Language


Title	Low-dimensional Embodied Semantics for Music and Language
Authors	Francisco Afonso Raposo, David Martins de Matos, Ricardo Ribeiro
Abstract	Embodied cognition states that semantics is encoded in the brain as firing patterns of neural circuits, which are learned according to the statistical structure of human multimodal experience. However, each human brain is idiosyncratically biased, according to its subjective experience history, making this biological semantic machinery noisy with respect to the overall semantics inherent to media artifacts, such as music and language excerpts. We propose to represent shared semantics using low-dimensional vector embeddings by jointly modeling several brains from human subjects. We show these unsupervised efficient representations outperform the original high-dimensional fMRI voxel spaces in proxy music genre and language topic classification tasks. We further show that joint modeling of several subjects increases the semantic richness of the learned latent vector spaces.
Tasks
Published	2019-06-20
URL	https://arxiv.org/abs/1906.11759v1
PDF	https://arxiv.org/pdf/1906.11759v1.pdf
PWC	https://paperswithcode.com/paper/low-dimensional-embodied-semantics-for-music
Repo
Framework

Deep Siamese Multi-scale Convolutional Network for Change Detection in Multi-temporal VHR Images


Title	Deep Siamese Multi-scale Convolutional Network for Change Detection in Multi-temporal VHR Images
Authors	Hongruixuan Chen, Chen Wu, Bo Du, Liangpei Zhang
Abstract	Very high resolution (VHR) images provide abundant ground details and spatial distribution information. Change detection in multi-temporal VHR images plays a significant role in urban expansion and area internal change analysis. Nevertheless, traditional change detection methods can neither take full advantage of spatial context information nor cope with the complex internal heterogeneity of VHR images. In this paper, we propose a powerful multi-scale feature convolution unit (MFCU) for change detection in VHR images. The proposed unit is able to extract multi-scale features in the same layer. Based on the proposed unit, two novel deep Siamese convolutional networks, deep Siamese multi-scale convolutional network (DSMS-CN) and deep Siamese multi-scale fully convolutional network (DSMS-FCN), are designed for unsupervised and supervised change detection in multi-temporal VHR images. For unsupervised change detection, we implement automatic pre-classification to obtain training patch samples, and the DSMS-CN fits the statistical distribution of changed and unchanged area from patch samples through multi-scale feature extraction module and deep Siamese architecture. For supervised change detection, the end-to-end deep fully convolutional network DSMS-FCN is trained in any size of multi-temporal VHR images, and directly output the binary change map. In addition, for the purpose of solving the inaccurate localization problem, the fully connected conditional random field (FC-CRF) is combined with DSMS-FCN to refine the results. The experimental results with challenging data sets confirm that the two proposed architectures perform better than the state-of-the-art methods.
Tasks
Published	2019-06-27
URL	https://arxiv.org/abs/1906.11479v1
PDF	https://arxiv.org/pdf/1906.11479v1.pdf
PWC	https://paperswithcode.com/paper/deep-siamese-multi-scale-convolutional
Repo
Framework

Technical report on Conversational Question Answering


Title	Technical report on Conversational Question Answering
Authors	Ying Ju, Fubang Zhao, Shijie Chen, Bowen Zheng, Xuefeng Yang, Yunfeng Liu
Abstract	Conversational Question Answering is a challenging task since it requires understanding of conversational history. In this project, we propose a new system RoBERTa + AT +KD, which involves rationale tagging multi-task, adversarial training, knowledge distillation and a linguistic post-process strategy. Our single model achieves 90.4(F1) on the CoQA test set without data augmentation, outperforming the current state-of-the-art single model by 2.6% F1.
Tasks	Data Augmentation, Question Answering
Published	2019-09-24
URL	https://arxiv.org/abs/1909.10772v1
PDF	https://arxiv.org/pdf/1909.10772v1.pdf
PWC	https://paperswithcode.com/paper/technical-report-on-conversational-question
Repo
Framework

Convolutional Quantum-Like Language Model with Mutual-Attention for Product Rating Prediction


Title	Convolutional Quantum-Like Language Model with Mutual-Attention for Product Rating Prediction
Authors	Qing Ping, Chaomei Chen
Abstract	Recommender systems are designed to help mitigate information overload users experience during online shopping. Recent work explores neural language models to learn user and item representations from user reviews and combines such representations with rating information. Most existing convolutional-based neural models take pooling immediately after convolution and loses the interaction information between the latent dimension of convolutional feature vectors along the way. Moreover, these models usually take all feature vectors at higher levels as equal and do not take into consideration that some features are more relevant to this specific user-item context. To bridge these gaps, this paper proposes a convolutional quantum-like language model with mutual-attention for rating prediction (ConQAR). By introducing a quantum-like density matrix layer, interactions between latent dimensions of convolutional feature vectors are well captured. With the attention weights learned from the mutual-attention layer, final representations of a user and an item absorb information from both itself and its counterparts for making rating prediction. Experiments on two large datasets show that our model outperforms multiple state-of-the-art CNN-based models. We also perform an ablation test to analyze the independent effects of the two components of our model. Moreover, we conduct a case study and present visualizations of the quantum probabilistic distributions in one user and one item review document to show that the learned distributions capture meaningful information about this user and item, and can be potentially used as textual profiling of the user and item.
Tasks	Language Modelling, Recommendation Systems
Published	2019-12-25
URL	https://arxiv.org/abs/1912.11720v1
PDF	https://arxiv.org/pdf/1912.11720v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-quantum-like-language-model
Repo
Framework

SECRET: Semantically Enhanced Classification of Real-world Tasks


Title	SECRET: Semantically Enhanced Classification of Real-world Tasks
Authors	Ayten Ozge Akmandor, Jorge Ortiz, Irene Manotas, Bongjun Ko, Niraj K. Jha
Abstract	Supervised machine learning (ML) algorithms are aimed at maximizing classification performance under available energy and storage constraints. They try to map the training data to the corresponding labels while ensuring generalizability to unseen data. However, they do not integrate meaning-based relationships among labels in the decision process. On the other hand, natural language processing (NLP) algorithms emphasize the importance of semantic information. In this paper, we synthesize the complementary advantages of supervised ML and NLP algorithms into one method that we refer to as SECRET (Semantically Enhanced Classification of REal-world Tasks). SECRET performs classifications by fusing the semantic information of the labels with the available data: it combines the feature space of the supervised algorithms with the semantic space of the NLP algorithms and predicts labels based on this joint space. Experimental results indicate that, compared to traditional supervised learning, SECRET achieves up to 14.0% accuracy and 13.1% F1 score improvements. Moreover, compared to ensemble methods, SECRET achieves up to 12.7% accuracy and 13.3% F1 score improvements. This points to a new research direction for supervised classification based on incorporation of semantic information.
Tasks
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12356v2
PDF	https://arxiv.org/pdf/1905.12356v2.pdf
PWC	https://paperswithcode.com/paper/secret-semantically-enhanced-classification
Repo
Framework

An Explainable Autoencoder For Collaborative Filtering Recommendation


Title	An Explainable Autoencoder For Collaborative Filtering Recommendation
Authors	Pegah Sagheb Haghighi, Olurotimi Seton, Olfa Nasraoui
Abstract	Autoencoders are a common building block of Deep Learning architectures, where they are mainly used for representation learning. They have also been successfully used in Collaborative Filtering (CF) recommender systems to predict missing ratings. Unfortunately, like all black box machine learning models, they are unable to explain their outputs. Hence, while predictions from an Autoencoder-based recommender system might be accurate, it might not be clear to the user why a recommendation was generated. In this work, we design an explainable recommendation system using an Autoencoder model whose predictions can be explained using the neighborhood based explanation style. Our preliminary work can be considered to be the first step towards an explainable deep learning architecture based on Autoencoders.
Tasks	Recommendation Systems, Representation Learning
Published	2019-12-23
URL	https://arxiv.org/abs/2001.04344v1
PDF	https://arxiv.org/pdf/2001.04344v1.pdf
PWC	https://paperswithcode.com/paper/an-explainable-autoencoder-for-collaborative
Repo
Framework

eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing


Title	eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing
Authors	Haoran Zhang, Ahmed Magooda, Diane Litman, Richard Correnti, Elaine Wang, Lindsay Clare Matsumura, Emily Howe, Rafael Quintana
Abstract	Writing a good essay typically involves students revising an initial paper draft after receiving feedback. We present eRevise, a web-based writing and revising environment that uses natural language processing features generated for rubric-based essay scoring to trigger formative feedback messages regarding students’ use of evidence in response-to-text writing. By helping students understand the criteria for using text evidence during writing, eRevise empowers students to better revise their paper drafts. In a pilot deployment of eRevise in 7 classrooms spanning grades 5 and 6, the quality of text evidence usage in writing improved after students received formative feedback then engaged in paper revision.
Tasks
Published	2019-08-06
URL	https://arxiv.org/abs/1908.01992v1
PDF	https://arxiv.org/pdf/1908.01992v1.pdf
PWC	https://paperswithcode.com/paper/erevise-using-natural-language-processing-to
Repo
Framework

Deep Elastic Networks with Model Selection for Multi-Task Learning


Title	Deep Elastic Networks with Model Selection for Multi-Task Learning
Authors	Chanho Ahn, Eunwoo Kim, Songhwai Oh
Abstract	In this work, we consider the problem of instance-wise dynamic network model selection for multi-task learning. To this end, we propose an efficient approach to exploit a compact but accurate model in a backbone architecture for each instance of all tasks. The proposed method consists of an estimator and a selector. The estimator is based on a backbone architecture and structured hierarchically. It can produce multiple different network models of different configurations in a hierarchical structure. The selector chooses a model dynamically from a pool of candidate models given an input instance. The selector is a relatively small-size network consisting of a few layers, which estimates a probability distribution over the candidate models when an input instance of a task is given. Both estimator and selector are jointly trained in a unified learning framework in conjunction with a sampling-based learning strategy, without additional computation steps. We demonstrate the proposed approach for several image classification tasks compared to existing approaches performing model selection or learning multiple tasks. Experimental results show that our approach gives not only outstanding performance compared to other competitors but also the versatility to perform instance-wise model selection for multiple tasks.
Tasks	Image Classification, Model Selection, Multi-Task Learning
Published	2019-09-11
URL	https://arxiv.org/abs/1909.04860v1
PDF	https://arxiv.org/pdf/1909.04860v1.pdf
PWC	https://paperswithcode.com/paper/deep-elastic-networks-with-model-selection
Repo
Framework

Nonregular and Minimax Estimation of Individualized Thresholds in High Dimension with Binary Responses


Title	Nonregular and Minimax Estimation of Individualized Thresholds in High Dimension with Binary Responses
Authors	Huijie Feng, Yang Ning, Jiwei Zhao
Abstract	Given a large number of covariates $Z$, we consider the estimation of a high-dimensional parameter $\theta$ in an individualized linear threshold $\theta^T Z$ for a continuous variable $X$, which minimizes the disagreement between $\text{sign}(X-\theta^TZ)$ and a binary response $Y$. While the problem can be formulated into the M-estimation framework, minimizing the corresponding empirical risk function is computationally intractable due to discontinuity of the sign function. Moreover, estimating $\theta$ even in the fixed-dimensional setting is known as a nonregular problem leading to nonstandard asymptotic theory. To tackle the computational and theoretical challenges in the estimation of the high-dimensional parameter $\theta$, we propose an empirical risk minimization approach based on a regularized smoothed loss function. The statistical and computational trade-off of the algorithm is investigated. Statistically, we show that the finite sample error bound for estimating $\theta$ in $\ell_2$ norm is $(s\log d/n)^{\beta/(2\beta+1)}$, where $d$ is the dimension of $\theta$, $s$ is the sparsity level, $n$ is the sample size and $\beta$ is the smoothness of the conditional density of $X$ given the response $Y$ and the covariates $Z$. The convergence rate is nonstandard and slower than that in the classical Lasso problems. Furthermore, we prove that the resulting estimator is minimax rate optimal up to a logarithmic factor. The Lepski’s method is developed to achieve the adaption to the unknown sparsity $s$ and smoothness $\beta$. Computationally, an efficient path-following algorithm is proposed to compute the solution path. We show that this algorithm achieves geometric rate of convergence for computing the whole path. Finally, we evaluate the finite sample performance of the proposed estimator in simulation studies and a real data analysis.
Tasks
Published	2019-05-26
URL	https://arxiv.org/abs/1905.10888v1
PDF	https://arxiv.org/pdf/1905.10888v1.pdf
PWC	https://paperswithcode.com/paper/nonregular-and-minimax-estimation-of
Repo
Framework

On Stabilizing Generative Adversarial Training with Noise


Title	On Stabilizing Generative Adversarial Training with Noise
Authors	Simon Jenni, Paolo Favaro
Abstract	We present a novel method and analysis to train generative adversarial networks (GAN) in a stable manner. As shown in recent analysis, training is often undermined by the probability distribution of the data being zero on neighborhoods of the data space. We notice that the distributions of real and generated data should match even when they undergo the same filtering. Therefore, to address the limited support problem we propose to train GANs by using different filtered versions of the real and generated data distributions. In this way, filtering does not prevent the exact matching of the data distribution, while helping training by extending the support of both distributions. As filtering we consider adding samples from an arbitrary distribution to the data, which corresponds to a convolution of the data distribution with the arbitrary one. We also propose to learn the generation of these samples so as to challenge the discriminator in the adversarial training. We show that our approach results in a stable and well-behaved training of even the original minimax GAN formulation. Moreover, our technique can be incorporated in most modern GAN formulations and leads to a consistent improvement on several common datasets.
Tasks
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04612v2
PDF	https://arxiv.org/pdf/1906.04612v2.pdf
PWC	https://paperswithcode.com/paper/on-stabilizing-generative-adversarial-1
Repo
Framework