October 18, 2019

3182 words 15 mins read

Paper Group ANR 423

Towards one-shot learning for rare-word translation with external experts. Deep Versus Wide Convolutional Neural Networks for Object Recognition on Neuromorphic System. Flexible Attributed Network Embedding. GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection. Transfer Learning …

Towards one-shot learning for rare-word translation with external experts


Title	Towards one-shot learning for rare-word translation with external experts
Authors	Ngoc-Quan Pham, Jan Niehues, Alex Waibel
Abstract	Neural machine translation (NMT) has significantly improved the quality of automatic translation models. One of the main challenges in current systems is the translation of rare words. We present a generic approach to address this weakness by having external models annotate the training data as Experts, and control the model-expert interaction with a pointer network and reinforcement learning. Our experiments using phrase-based models to simulate Experts to complement neural machine translation models show that the model can be trained to copy the annotations into the output consistently. We demonstrate the benefit of our proposed framework in outof-domain translation scenarios with only lexical resources, improving more than 1.0 BLEU point in both translation directions English to Spanish and German to English
Tasks	Machine Translation, One-Shot Learning
Published	2018-09-10
URL	http://arxiv.org/abs/1809.03182v1
PDF	http://arxiv.org/pdf/1809.03182v1.pdf
PWC	https://paperswithcode.com/paper/towards-one-shot-learning-for-rare-word
Repo
Framework

Deep Versus Wide Convolutional Neural Networks for Object Recognition on Neuromorphic System


Title	Deep Versus Wide Convolutional Neural Networks for Object Recognition on Neuromorphic System
Authors	Md Zahangir Alom, Theodore Josue, Md Nayim Rahman, Will Mitchell, Chris Yakopcic, Tarek M. Taha
Abstract	In the last decade, special purpose computing systems, such as Neuromorphic computing, have become very popular in the field of computer vision and machine learning for classification tasks. In 2015, IBM’s released the TrueNorth Neuromorphic system, kick-starting a new era of Neuromorphic computing. Alternatively, Deep Learning approaches such as Deep Convolutional Neural Networks (DCNN) show almost human-level accuracies for detection and classification tasks. IBM’s 2016 release of a deep learning framework for DCNNs, called Energy Efficient Deep Neuromorphic Networks (Eedn). Eedn shows promise for delivering high accuracies across a number of different benchmarks, while consuming very low power, using IBM’s TrueNorth chip. However, there are many things that remained undiscovered using the Eedn framework for classification tasks on a Neuromorphic system. In this paper, we have empirically evaluated the performance of different DCNN architectures implemented within the Eedn framework. The goal of this work was discover the most efficient way to implement DCNN models for object classification tasks using the TrueNorth system. We performed our experiments using benchmark data sets such as MNIST, COIL 20, and COIL 100. The experimental results show very promising classification accuracies with very low power consumption on IBM’s NS1e Neurosynaptic system. The results show that for datasets with large numbers of classes, wider networks perform better when compared to deep networks comprised of nearly the same core complexity on IBM’s TrueNorth system.
Tasks	Object Classification, Object Recognition
Published	2018-02-07
URL	http://arxiv.org/abs/1802.02608v1
PDF	http://arxiv.org/pdf/1802.02608v1.pdf
PWC	https://paperswithcode.com/paper/deep-versus-wide-convolutional-neural
Repo
Framework

Flexible Attributed Network Embedding


Title	Flexible Attributed Network Embedding
Authors	Enya Shen, Zhidong Cao, Changqing Zou, Jianmin Wang
Abstract	Network embedding aims to find a way to encode network by learning an embedding vector for each node in the network. The network often has property information which is highly informative with respect to the node’s position and role in the network. Most network embedding methods fail to utilize this information during network representation learning. In this paper, we propose a novel framework, FANE, to integrate structure and property information in the network embedding process. In FANE, we design a network to unify heterogeneity of the two information sources, and define a new random walking strategy to leverage property information and make the two information compensate. FANE is conceptually simple and empirically powerful. It improves over the state-of-the-art methods on Cora dataset classification task by over 5%, more than 10% on WebKB dataset classification task. Experiments also show that the results improve more than the state-of-the-art methods as increasing training size. Moreover, qualitative visualization show that our framework is helpful in network property information exploration. In all, we present a new way for efficiently learning state-of-the-art task-independent representations in complex attributed networks. The source code and datasets of this paper can be obtained from https://github.com/GraphWorld/FANE.
Tasks	Network Embedding, Representation Learning
Published	2018-11-27
URL	http://arxiv.org/abs/1811.10789v1
PDF	http://arxiv.org/pdf/1811.10789v1.pdf
PWC	https://paperswithcode.com/paper/flexible-attributed-network-embedding
Repo
Framework

GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection


Title	GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection
Authors	Egor Lakomkin, Chandrakant Bothe, Stefan Wermter
Abstract	The WASSA 2017 EmoInt shared task has the goal to predict emotion intensity values of tweet messages. Given the text of a tweet and its emotion category (anger, joy, fear, and sadness), the participants were asked to build a system that assigns emotion intensity values. Emotion intensity estimation is a challenging problem given the short length of the tweets, the noisy structure of the text and the lack of annotated data. To solve this problem, we developed an ensemble of two neural models, processing input on the character. and word-level with a lexicon-driven system. The correlation scores across all four emotions are averaged to determine the bottom-line competition metric, and our system ranks place forth in full intensity range and third in 0.5-1 range of intensity among 23 systems at the time of writing (June 2017).
Tasks
Published	2018-03-30
URL	http://arxiv.org/abs/1803.11509v1
PDF	http://arxiv.org/pdf/1803.11509v1.pdf
PWC	https://paperswithcode.com/paper/gradascent-at-emoint-2017-character-and-word
Repo
Framework

Transfer Learning in Astronomy: A New Machine-Learning Paradigm


Title	Transfer Learning in Astronomy: A New Machine-Learning Paradigm
Authors	Ricardo Vilalta
Abstract	The widespread dissemination of machine learning tools in science, particularly in astronomy, has revealed the limitation of working with simple single-task scenarios in which any task in need of a predictive model is looked in isolation, and ignores the existence of other similar tasks. In contrast, a new generation of techniques is emerging where predictive models can take advantage of previous experience to leverage information from similar tasks. The new emerging area is referred to as transfer learning. In this paper, I briefly describe the motivation behind the use of transfer learning techniques, and explain how such techniques can be used to solve popular problems in astronomy. As an example, a prevalent problem in astronomy is to estimate the class of an object (e.g., Supernova Ia) using a generation of photometric light-curve datasets where data abounds, but class labels are scarce; such analysis can benefit from spectroscopic data where class labels are known with high confidence, but the data sample is small. Transfer learning provides a robust and practical solution to leverage information from one domain to improve the accuracy of a model built on a different domain. In the example above, transfer learning would look to overcome the difficulty in the compatibility of models between spectroscopic data and photometric data, since data properties such as size, class priors, and underlying distributions, are all expected to be significantly different.
Tasks	Transfer Learning
Published	2018-12-20
URL	http://arxiv.org/abs/1812.10403v1
PDF	http://arxiv.org/pdf/1812.10403v1.pdf
PWC	https://paperswithcode.com/paper/transfer-learning-in-astronomy-a-new-machine
Repo
Framework

Sequential Maximum Margin Classifiers for Partially Labeled Data


Title	Sequential Maximum Margin Classifiers for Partially Labeled Data
Authors	Elizabeth Hou, Alfred O. Hero
Abstract	In many real-world applications, data is not collected as one batch, but sequentially over time, and often it is not possible or desirable to wait until the data is completely gathered before analyzing it. Thus, we propose a framework to sequentially update a maximum margin classifier by taking advantage of the Maximum Entropy Discrimination principle. Our maximum margin classifier allows for a kernel representation to represent large numbers of features and can also be regularized with respect to a smooth sub-manifold, allowing it to incorporate unlabeled observations. We compare the performance of our classifier to its non-sequential equivalents in both simulated and real datasets.
Tasks
Published	2018-03-07
URL	http://arxiv.org/abs/1803.02517v1
PDF	http://arxiv.org/pdf/1803.02517v1.pdf
PWC	https://paperswithcode.com/paper/sequential-maximum-margin-classifiers-for
Repo
Framework

Feedback GAN (FBGAN) for DNA: a Novel Feedback-Loop Architecture for Optimizing Protein Functions


Title	Feedback GAN (FBGAN) for DNA: a Novel Feedback-Loop Architecture for Optimizing Protein Functions
Authors	Anvita Gupta, James Zou
Abstract	Generative Adversarial Networks (GANs) represent an attractive and novel approach to generate realistic data, such as genes, proteins, or drugs, in synthetic biology. Here, we apply GANs to generate synthetic DNA sequences encoding for proteins of variable length. We propose a novel feedback-loop architecture, called Feedback GAN (FBGAN), to optimize the synthetic gene sequences for desired properties using an external function analyzer. The proposed architecture also has the advantage that the analyzer need not be differentiable. We apply the feedback-loop mechanism to two examples: 1) generating synthetic genes coding for antimicrobial peptides, and 2) optimizing synthetic genes for the secondary structure of their resulting peptides. A suite of metrics demonstrate that the GAN generated proteins have desirable biophysical properties. The FBGAN architecture can also be used to optimize GAN-generated datapoints for useful properties in domains beyond genomics.
Tasks
Published	2018-04-05
URL	http://arxiv.org/abs/1804.01694v1
PDF	http://arxiv.org/pdf/1804.01694v1.pdf
PWC	https://paperswithcode.com/paper/feedback-gan-fbgan-for-dna-a-novel-feedback
Repo
Framework

Semi-supervised Deep Reinforcement Learning in Support of IoT and Smart City Services


Title	Semi-supervised Deep Reinforcement Learning in Support of IoT and Smart City Services
Authors	Mehdi Mohammadi, Ala Al-Fuqaha, Mohsen Guizani, Jun-Seok Oh
Abstract	Smart services are an important element of the smart cities and the Internet of Things (IoT) ecosystems where the intelligence behind the services is obtained and improved through the sensory data. Providing a large amount of training data is not always feasible; therefore, we need to consider alternative ways that incorporate unlabeled data as well. In recent years, Deep reinforcement learning (DRL) has gained great success in several application domains. It is an applicable method for IoT and smart city scenarios where auto-generated data can be partially labeled by users’ feedback for training purposes. In this paper, we propose a semi-supervised deep reinforcement learning model that fits smart city applications as it consumes both labeled and unlabeled data to improve the performance and accuracy of the learning agent. The model utilizes Variational Autoencoders (VAE) as the inference engine for generalizing optimal policies. To the best of our knowledge, the proposed model is the first investigation that extends deep reinforcement learning to the semi-supervised paradigm. As a case study of smart city applications, we focus on smart buildings and apply the proposed model to the problem of indoor localization based on BLE signal strength. Indoor localization is the main component of smart city services since people spend significant time in indoor environments. Our model learns the best action policies that lead to a close estimation of the target locations with an improvement of 23% in terms of distance to the target and at least 67% more received rewards compared to the supervised DRL model.
Tasks
Published	2018-10-09
URL	http://arxiv.org/abs/1810.04118v1
PDF	http://arxiv.org/pdf/1810.04118v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-deep-reinforcement-learning
Repo
Framework

CoverBLIP: scalable iterative matched filtering for MR Fingerprint recovery


Title	CoverBLIP: scalable iterative matched filtering for MR Fingerprint recovery
Authors	Mohammad Golbabaee, Zhouye Chen, Yves Wiaux, Mike E. Davies
Abstract	Current proposed solutions for the high dimensionality of the MRF reconstruction problem rely on a linear compression step to reduce the matching computations and boost the efficiency of fast but non-scalable searching schemes such as the KD-trees. However such methodologies often introduce an unfavourable compromise in the estimation accuracy when applied to nonlinear data structures such as the manifold of Bloch responses with possible increased dynamic complexity and growth in data population. To address this shortcoming we propose an inexact iterative reconstruction method, dubbed as the Cover BLoch response Iterative Projection (CoverBLIP). Iterative methods improve the accuracy of their non-iterative counterparts and are additionally robust against certain accelerated approximate updates, without compromising their final accuracy. Leveraging on these results, we accelerate matched-filtering using an ANNS algorithm based on Cover trees with a robustness feature against the curse of dimensionality.
Tasks
Published	2018-09-06
URL	http://arxiv.org/abs/1809.02503v1
PDF	http://arxiv.org/pdf/1809.02503v1.pdf
PWC	https://paperswithcode.com/paper/coverblip-scalable-iterative-matched
Repo
Framework

Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech


Title	Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech
Authors	David Harwath, Galen Chuang, James Glass
Abstract	In this paper, we explore the learning of neural network embeddings for natural images and speech waveforms describing the content of those images. These embeddings are learned directly from the waveforms without the use of linguistic transcriptions or conventional speech recognition technology. While prior work has investigated this setting in the monolingual case using English speech data, this work represents the first effort to apply these techniques to languages beyond English. Using spoken captions collected in English and Hindi, we show that the same model architecture can be successfully applied to both languages. Further, we demonstrate that training a multilingual model simultaneously on both languages offers improved performance over the monolingual models. Finally, we show that these models are capable of performing semantic cross-lingual speech-to-speech retrieval.
Tasks	Speech Recognition
Published	2018-04-09
URL	http://arxiv.org/abs/1804.03052v1
PDF	http://arxiv.org/pdf/1804.03052v1.pdf
PWC	https://paperswithcode.com/paper/vision-as-an-interlingua-learning
Repo
Framework

Characterizing Design Patterns of EHR-Driven Phenotype Extraction Algorithms


Title	Characterizing Design Patterns of EHR-Driven Phenotype Extraction Algorithms
Authors	Yizhen Zhong, Luke Rasmussen, Yu Deng, Jennifer Pacheco, Maureen Smith, Justin Starren, Wei-Qi Wei, Peter Speltz, Joshua Denny, Nephi Walton, George Hripcsak, Christopher G Chute, Yuan Luo
Abstract	The automatic development of phenotype algorithms from Electronic Health Record data with machine learning (ML) techniques is of great interest given the current practice is very time-consuming and resource intensive. The extraction of design patterns from phenotype algorithms is essential to understand their rationale and standard, with great potential to automate the development process. In this pilot study, we perform network visualization on the design patterns and their associations with phenotypes and sites. We classify design patterns using the fragments from previously annotated phenotype algorithms as the ground truth. The classification performance is used as a proxy for coherence at the attribution level. The bag-of-words representation with knowledge-based features generated a good performance in the classification task (0.79 macro-f1 scores). Good classification accuracy with simple features demonstrated the attribution coherence and the feasibility of automatic identification of design patterns. Our results point to both the feasibility and challenges of automatic identification of phenotyping design patterns, which would power the automatic development of phenotype algorithms.
Tasks
Published	2018-11-15
URL	http://arxiv.org/abs/1811.06183v1
PDF	http://arxiv.org/pdf/1811.06183v1.pdf
PWC	https://paperswithcode.com/paper/characterizing-design-patterns-of-ehr-driven
Repo
Framework

SAR Image Despeckling Using Quadratic-Linear Approximated L1-Norm


Title	SAR Image Despeckling Using Quadratic-Linear Approximated L1-Norm
Authors	Fatih Nar
Abstract	Speckle noise, inherent in synthetic aperture radar (SAR) images, degrades the performance of the various SAR image analysis tasks. Thus, speckle noise reduction is a critical preprocessing step for smoothing homogeneous regions while preserving details. This letter proposes a variational despeckling approach where L1-norm total variation regularization term is approximated in a quadratic and linear manner to increase accuracy while decreasing the computation time. Despeckling performance and computational efficiency of the proposed method are shown using synthetic and real-world SAR images.
Tasks	Sar Image Despeckling
Published	2018-01-15
URL	http://arxiv.org/abs/1801.04751v1
PDF	http://arxiv.org/pdf/1801.04751v1.pdf
PWC	https://paperswithcode.com/paper/sar-image-despeckling-using-quadratic-linear
Repo
Framework

Generative Models for Simulating Mobility Trajectories


Title	Generative Models for Simulating Mobility Trajectories
Authors	Vaibhav Kulkarni, Natasa Tagasovska, Thibault Vatter, Benoit Garbinato
Abstract	Mobility datasets are fundamental for evaluating algorithms pertaining to geographic information systems and facilitating experimental reproducibility. But privacy implications restrict sharing such datasets, as even aggregated location-data is vulnerable to membership inference attacks. Current synthetic mobility dataset generators attempt to superficially match a priori modeled mobility characteristics which do not accurately reflect the real-world characteristics. Modeling human mobility to generate synthetic yet semantically and statistically realistic trajectories is therefore crucial for publishing trajectory datasets having satisfactory utility level while preserving user privacy. Specifically, long-range dependencies inherent to human mobility are challenging to capture with both discriminative and generative models. In this paper, we benchmark the performance of recurrent neural architectures (RNNs), generative adversarial networks (GANs) and nonparametric copulas to generate synthetic mobility traces. We evaluate the generated trajectories with respect to their geographic and semantic similarity, circadian rhythms, long-range dependencies, training and generation time. We also include two sample tests to assess statistical similarity between the observed and simulated distributions, and we analyze the privacy tradeoffs with respect to membership inference and location-sequence attacks.
Tasks	Semantic Similarity, Semantic Textual Similarity
Published	2018-11-30
URL	http://arxiv.org/abs/1811.12801v1
PDF	http://arxiv.org/pdf/1811.12801v1.pdf
PWC	https://paperswithcode.com/paper/generative-models-for-simulating-mobility
Repo
Framework

Pixel and Feature Level Based Domain Adaption for Object Detection in Autonomous Driving


Title	Pixel and Feature Level Based Domain Adaption for Object Detection in Autonomous Driving
Authors	Yuhu Shan, Wen Feng Lu, Chee Meng Chew
Abstract	Annotating large scale datasets to train modern convolutional neural networks is prohibitively expensive and time-consuming for many real tasks. One alternative is to train the model on labeled synthetic datasets and apply it in the real scenes. However, this straightforward method often fails to generalize well mainly due to the domain bias between the synthetic and real datasets. Many unsupervised domain adaptation (UDA) methods are introduced to address this problem but most of them only focus on the simple classification task. In this paper, we present a novel UDA model to solve the more complex object detection problem in the context of autonomous driving. Our model integrates both pixel level and feature level based transformtions to fulfill the cross domain detection task and can be further trained end-to-end to pursue better performance. We employ objectives of the generative adversarial network and the cycle consistency loss for image translation in the pixel space. To address the potential semantic inconsistency problem, we propose region proposal based feature adversarial training to preserve the semantics of our target objects as well as further minimize the domain shifts. Extensive experiments are conducted on several different datasets, and the results demonstrate the robustness and superiority of our method.
Tasks	Autonomous Driving, Domain Adaptation, Object Detection, Unsupervised Domain Adaptation
Published	2018-09-30
URL	https://arxiv.org/abs/1810.00345v2
PDF	https://arxiv.org/pdf/1810.00345v2.pdf
PWC	https://paperswithcode.com/paper/pixel-and-feature-level-based-domain-adaption
Repo
Framework

Automata for Infinite Argumentation Structures


Title	Automata for Infinite Argumentation Structures
Authors	Pietro Baroni, Federico Cerutti, Paul E. Dunne, Massimiliano Giacomin
Abstract	The theory of abstract argumentation frameworks (afs) has, in the main, focused on finite structures, though there are many significant contexts where argumentation can be regarded as a process involving infinite objects. To address this limitation, in this paper we propose a novel approach for describing infinite afs using tools from formal language theory. In particular, the possibly infinite set of arguments is specified through the language recognized by a deterministic finite automaton while a suitable formalism, called attack expression, is introduced to describe the relation of attack between arguments. The proposed approach is shown to satisfy some desirable properties which can not be achieved through other “naive” uses of formal languages. In particular, the approach is shown to be expressive enough to capture (besides any arbitrary finite structure) a large variety of infinite afs including two major examples from previous literature and two sample cases from the domains of multi-agent negotiation and ambient intelligence. On the computational side, we show that several decision and construction problems which are known to be polynomial time solvable in finite afs are decidable in the context of the proposed formalism and we provide the relevant algorithms. Moreover we obtain additional results concerning the case of finitary afs.
Tasks	Abstract Argumentation
Published	2018-10-11
URL	http://arxiv.org/abs/1810.04892v1
PDF	http://arxiv.org/pdf/1810.04892v1.pdf
PWC	https://paperswithcode.com/paper/automata-for-infinite-argumentation
Repo
Framework