October 18, 2019

2902 words 14 mins read

Paper Group ANR 410

Can VAEs Generate Novel Examples?. Towards end-to-end spoken language understanding. Fine Tuning Method by using Knowledge Acquisition from Deep Belief Network. Adaptive Sensing for Learning Nonstationary Environment Models. Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation. Preparing Bengali-Eng …

Can VAEs Generate Novel Examples?


Title	Can VAEs Generate Novel Examples?
Authors	Alican Bozkurt, Babak Esmaeili, Dana H. Brooks, Jennifer G. Dy, Jan-Willem van de Meent
Abstract	An implicit goal in works on deep generative models is that such models should be able to generate novel examples that were not previously seen in the training data. In this paper, we investigate to what extent this property holds for widely employed variational autoencoder (VAE) architectures. VAEs maximize a lower bound on the log marginal likelihood, which implies that they will in principle overfit the training data when provided with a sufficiently expressive decoder. In the limit of an infinite capacity decoder, the optimal generative model is a uniform mixture over the training data. More generally, an optimal decoder should output a weighted average over the examples in the training data, where the magnitude of the weights is determined by the proximity in the latent space. This leads to the hypothesis that, for a sufficiently high capacity encoder and decoder, the VAE decoder will perform nearest-neighbor matching according to the coordinates in the latent space. To test this hypothesis, we investigate generalization on the MNIST dataset. We consider both generalization to new examples of previously seen classes, and generalization to the classes that were withheld from the training set. In both cases, we find that reconstructions are closely approximated by nearest neighbors for higher-dimensional parameterizations. When generalizing to unseen classes however, lower-dimensional parameterizations offer a clear advantage.
Tasks
Published	2018-12-22
URL	http://arxiv.org/abs/1812.09624v1
PDF	http://arxiv.org/pdf/1812.09624v1.pdf
PWC	https://paperswithcode.com/paper/can-vaes-generate-novel-examples
Repo
Framework

Towards end-to-end spoken language understanding


Title	Towards end-to-end spoken language understanding
Authors	Dmitriy Serdyuk, Yongqiang Wang, Christian Fuegen, Anuj Kumar, Baiyang Liu, Yoshua Bengio
Abstract	Spoken language understanding system is traditionally designed as a pipeline of a number of components. First, the audio signal is processed by an automatic speech recognizer for transcription or n-best hypotheses. With the recognition results, a natural language understanding system classifies the text to structured data as domain, intent and slots for down-streaming consumers, such as dialog system, hands-free applications. These components are usually developed and optimized independently. In this paper, we present our study on an end-to-end learning system for spoken language understanding. With this unified approach, we can infer the semantic meaning directly from audio features without the intermediate text representation. This study showed that the trained model can achieve reasonable good result and demonstrated that the model can capture the semantic attention directly from the audio features.
Tasks	Spoken Language Understanding
Published	2018-02-23
URL	http://arxiv.org/abs/1802.08395v1
PDF	http://arxiv.org/pdf/1802.08395v1.pdf
PWC	https://paperswithcode.com/paper/towards-end-to-end-spoken-language
Repo
Framework

Fine Tuning Method by using Knowledge Acquisition from Deep Belief Network


Title	Fine Tuning Method by using Knowledge Acquisition from Deep Belief Network
Authors	Shin Kamada, Takumi Ichimura
Abstract	We developed an adaptive structure learning method of Restricted Boltzmann Machine (RBM) which can generate/annihilate neurons by self-organizing learning method according to input patterns. Moreover, the adaptive Deep Belief Network (DBN) in the assemble process of pre-trained RBM layer was developed. The proposed method presents to score a great success to the training data set for big data benchmark test such as CIFAR-10. However, the classification capability of the test data set, which are included unknown patterns, is high, but does not lead perfect correct solution. We investigated the wrong specified data and then some characteristic patterns were found. In this paper, the knowledge related to the patterns is embedded into the classification algorithm of trained DBN. As a result, the classification capability can achieve a great success (97.1% to unknown data set).
Tasks
Published	2018-07-10
URL	http://arxiv.org/abs/1807.03487v2
PDF	http://arxiv.org/pdf/1807.03487v2.pdf
PWC	https://paperswithcode.com/paper/fine-tuning-method-by-using-knowledge
Repo
Framework

Adaptive Sensing for Learning Nonstationary Environment Models


Title	Adaptive Sensing for Learning Nonstationary Environment Models
Authors	Sahil Garg, Amarjeet Singh, Fabio Ramos
Abstract	Most environmental phenomena, such as wind profiles, ozone concentration and sunlight distribution under a forest canopy, exhibit nonstationary dynamics i.e. phenomenon variation change depending on the location and time of occurrence. Non-stationary dynamics pose both theoretical and practical challenges to statistical machine learning algorithms aiming to accurately capture the complexities governing the evolution of such processes. In this paper, we address the sampling aspects of the problem of learning nonstationary spatio-temporal models, and propose an efficient yet simple algorithm - LISAL. The core idea in LISAL is to learn two models using Gaussian processes (GPs) wherein the first is a nonstationary GP directly modeling the phenomenon. The second model uses a stationary GP representing a latent space corresponding to changes in dynamics, or the nonstationarity characteristics of the first model. LISAL involves adaptively sampling the latent space dynamics using information theory quantities to reduce the computational cost during the learning phase. The relevance of LISAL is extensively validated using multiple real world datasets.
Tasks	Gaussian Processes
Published	2018-04-26
URL	http://arxiv.org/abs/1804.10279v1
PDF	http://arxiv.org/pdf/1804.10279v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-sensing-for-learning-nonstationary
Repo
Framework

Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation


Title	Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation
Authors	Chaowei Xiao, Ruizhi Deng, Bo Li, Fisher Yu, Mingyan Liu, Dawn Song
Abstract	Deep Neural Networks (DNNs) have been widely applied in various recognition tasks. However, recently DNNs have been shown to be vulnerable against adversarial examples, which can mislead DNNs to make arbitrary incorrect predictions. While adversarial examples are well studied in classification tasks, other learning problems may have different properties. For instance, semantic segmentation requires additional components such as dilated convolutions and multiscale processing. In this paper, we aim to characterize adversarial examples based on spatial context information in semantic segmentation. We observe that spatial consistency information can be potentially leveraged to detect adversarial examples robustly even when a strong adaptive attacker has access to the model and detection strategies. We also show that adversarial examples based on attacks considered within the paper barely transfer among models, even though transferability is common in classification. Our observations shed new light on developing adversarial attacks and defenses to better understand the vulnerabilities of DNNs.
Tasks	Semantic Segmentation
Published	2018-10-11
URL	http://arxiv.org/abs/1810.05162v1
PDF	http://arxiv.org/pdf/1810.05162v1.pdf
PWC	https://paperswithcode.com/paper/characterizing-adversarial-examples-based-on
Repo
Framework

Preparing Bengali-English Code-Mixed Corpus for Sentiment Analysis of Indian Languages


Title	Preparing Bengali-English Code-Mixed Corpus for Sentiment Analysis of Indian Languages
Authors	Soumil Mandal, Sainik Kumar Mahata, Dipankar Das
Abstract	Analysis of informative contents and sentiments of social users has been attempted quite intensively in the recent past. Most of the systems are usable only for monolingual data and fails or gives poor results when used on data with code-mixing property. To gather attention and encourage researchers to work on this crisis, we prepared gold standard Bengali-English code-mixed data with language and polarity tag for sentiment analysis purposes. In this paper, we discuss the systems we prepared to collect and filter raw Twitter data. In order to reduce manual work while annotation, hybrid systems combining rule based and supervised models were developed for both language and sentiment tagging. The final corpus was annotated by a group of annotators following a few guidelines. The gold standard corpus thus obtained has impressive inter-annotator agreement obtained in terms of Kappa values. Various metrics like Code-Mixed Index (CMI), Code-Mixed Factor (CF) along with various aspects (language and emotion) also qualitatively polled the code-mixed and sentiment properties of the corpus.
Tasks	Sentiment Analysis
Published	2018-03-11
URL	http://arxiv.org/abs/1803.04000v1
PDF	http://arxiv.org/pdf/1803.04000v1.pdf
PWC	https://paperswithcode.com/paper/preparing-bengali-english-code-mixed-corpus
Repo
Framework

Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss


Title	Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss
Authors	Sangil Jung, Changyong Son, Seohyung Lee, Jinwoo Son, Youngjun Kwak, Jae-Joon Han, Sung Ju Hwang, Changkyu Choi
Abstract	Reducing bit-widths of activations and weights of deep networks makes it efficient to compute and store them in memory, which is crucial in their deployments to resource-limited devices, such as mobile phones. However, decreasing bit-widths with quantization generally yields drastically degraded accuracy. To tackle this problem, we propose to learn to quantize activations and weights via a trainable quantizer that transforms and discretizes them. Specifically, we parameterize the quantization intervals and obtain their optimal values by directly minimizing the task loss of the network. This quantization-interval-learning (QIL) allows the quantized networks to maintain the accuracy of the full-precision (32-bit) networks with bit-width as low as 4-bit and minimize the accuracy degeneration with further bit-width reduction (i.e., 3 and 2-bit). Moreover, our quantizer can be trained on a heterogeneous dataset, and thus can be used to quantize pretrained networks without access to their training data. We demonstrate the effectiveness of our trainable quantizer on ImageNet dataset with various network architectures such as ResNet-18, -34 and AlexNet, on which it outperforms existing methods to achieve the state-of-the-art accuracy.
Tasks	Quantization
Published	2018-08-17
URL	http://arxiv.org/abs/1808.05779v3
PDF	http://arxiv.org/pdf/1808.05779v3.pdf
PWC	https://paperswithcode.com/paper/learning-to-quantize-deep-networks-by
Repo
Framework

Lifted Filtering via Exchangeable Decomposition


Title	Lifted Filtering via Exchangeable Decomposition
Authors	Stefan Lüdtke, Max Schröder, Sebastian Bader, Kristian Kersting, Thomas Kirste
Abstract	We present a model for exact recursive Bayesian filtering based on lifted multiset states. Combining multisets with lifting makes it possible to simultaneously exploit multiple strategies for reducing inference complexity when compared to list-based grounded state representations. The core idea is to borrow the concept of Maximally Parallel Multiset Rewriting Systems and to enhance it by concepts from Rao-Blackwellization and Lifted Inference, giving a representation of state distributions that enables efficient inference. In worlds where the random variables that define the system state are exchangeable – where the identity of entities does not matter – it automatically uses a representation that abstracts from ordering (achieving an exponential reduction in complexity) – and it automatically adapts when observations or system dynamics destroy exchangeability by breaking symmetry.
Tasks
Published	2018-01-31
URL	http://arxiv.org/abs/1801.10495v2
PDF	http://arxiv.org/pdf/1801.10495v2.pdf
PWC	https://paperswithcode.com/paper/lifted-filtering-via-exchangeable
Repo
Framework

Learning Recurrent Binary/Ternary Weights


Title	Learning Recurrent Binary/Ternary Weights
Authors	Arash Ardakani, Zhengyun Ji, Sean C. Smithson, Brett H. Meyer, Warren J. Gross
Abstract	Recurrent neural networks (RNNs) have shown excellent performance in processing sequence data. However, they are both complex and memory intensive due to their recursive nature. These limitations make RNNs difficult to embed on mobile devices requiring real-time processes with limited hardware resources. To address the above issues, we introduce a method that can learn binary and ternary weights during the training phase to facilitate hardware implementations of RNNs. As a result, using this approach replaces all multiply-accumulate operations by simple accumulations, bringing significant benefits to custom hardware in terms of silicon area and power consumption. On the software side, we evaluate the performance (in terms of accuracy) of our method using long short-term memories (LSTMs) on various sequential models including sequence classification and language modeling. We demonstrate that our method achieves competitive results on the aforementioned tasks while using binary/ternary weights during the runtime. On the hardware side, we present custom hardware for accelerating the recurrent computations of LSTMs with binary/ternary weights. Ultimately, we show that LSTMs with binary/ternary weights can achieve up to 12x memory saving and 10x inference speedup compared to the full-precision implementation on an ASIC platform.
Tasks	Language Modelling
Published	2018-09-28
URL	http://arxiv.org/abs/1809.11086v2
PDF	http://arxiv.org/pdf/1809.11086v2.pdf
PWC	https://paperswithcode.com/paper/learning-recurrent-binaryternary-weights
Repo
Framework

Automatic Annotation of Locative and Directional Expressions in Arabic


Title	Automatic Annotation of Locative and Directional Expressions in Arabic
Authors	Rita Hijazi, Amani Sabra, Moustafa Al-Hajj
Abstract	In this paper, we introduce a rule-based approach to annotate Locative and Directional Expressions in Arabic natural language text. The annotation is based on a constructed semantic map of the spatiality domain. Challenges are twofold: first, we need to study how locative and directional expressions are expressed linguistically in these texts; and second, we need to automatically annotate the relevant textual segments accordingly. The research method we will use in this article is analytic-descriptive. We will validate this approach on specific novel rich with these expressions and show that it has very promising results. We will be using NOOJ as a software tool to implement finite-state transducers to annotate linguistic elements according to Locative and Directional Expressions. In conclusion, NOOJ allowed us to write linguistic rules for the automatic annotation in Arabic text of Locative and Directional Expressions.
Tasks
Published	2018-05-16
URL	http://arxiv.org/abs/1805.06344v2
PDF	http://arxiv.org/pdf/1805.06344v2.pdf
PWC	https://paperswithcode.com/paper/automatic-annotation-of-locative-and
Repo
Framework

Weakly-Supervised Localization and Classification of Proximal Femur Fractures


Title	Weakly-Supervised Localization and Classification of Proximal Femur Fractures
Authors	Amelia Jiménez-Sánchez, Anees Kazi, Shadi Albarqouni, Sonja Kirchhoff, Alexandra Sträter, Peter Biberthaler, Diana Mateus, Nassir Navab
Abstract	In this paper, we target the problem of fracture classification from clinical X-Ray images towards an automated Computer Aided Diagnosis (CAD) system. Although primarily dealing with an image classification problem, we argue that localizing the fracture in the image is crucial to make good class predictions. Therefore, we propose and thoroughly analyze several schemes for simultaneous fracture localization and classification. We show that using an auxiliary localization task, in general, improves the classification performance. Moreover, it is possible to avoid the need for additional localization annotations thanks to recent advancements in weakly-supervised deep learning approaches. Among such approaches, we investigate and adapt Spatial Transformers (ST), Self-Transfer Learning (STL), and localization from global pooling layers. We provide a detailed quantitative and qualitative validation on a dataset of 1347 femur fractures images and report high accuracy with regard to inter-expert correlation values reported in the literature. Our investigations show that i) lesion localization improves the classification outcome, ii) weakly-supervised methods improve baseline classification without any additional cost, iii) STL guides feature activations and boost performance. We plan to make both the dataset and code available.
Tasks	Image Classification, Transfer Learning
Published	2018-09-27
URL	http://arxiv.org/abs/1809.10692v1
PDF	http://arxiv.org/pdf/1809.10692v1.pdf
PWC	https://paperswithcode.com/paper/weakly-supervised-localization-and
Repo
Framework

Sentiment Index of the Russian Speaking Facebook


Title	Sentiment Index of the Russian Speaking Facebook
Authors	Alexander Panchenko
Abstract	A sentiment index measures the average emotional level in a corpus. We introduce four such indexes and use them to gauge average “positiveness” of a population during some period based on posts in a social network. This article for the first time presents a text-, rather than word-based sentiment index. Furthermore, this study presents the first large-scale study of the sentiment index of the Russian-speaking Facebook. Our results are consistent with the prior experiments for the English language.
Tasks
Published	2018-08-23
URL	http://arxiv.org/abs/1808.07851v1
PDF	http://arxiv.org/pdf/1808.07851v1.pdf
PWC	https://paperswithcode.com/paper/sentiment-index-of-the-russian-speaking
Repo
Framework

Robust Graph Learning from Noisy Data


Title	Robust Graph Learning from Noisy Data
Authors	Zhao Kang, Haiqi Pan, Steven C. H. Hoi, Zenglin Xu
Abstract	Learning graphs from data automatically has shown encouraging performance on clustering and semisupervised learning tasks. However, real data are often corrupted, which may cause the learned graph to be inexact or unreliable. In this paper, we propose a novel robust graph learning scheme to learn reliable graphs from real-world noisy data by adaptively removing noise and errors in the raw data. We show that our proposed model can also be viewed as a robust version of manifold regularized robust PCA, where the quality of the graph plays a critical role. The proposed model is able to boost the performance of data clustering, semisupervised classification, and data recovery significantly, primarily due to two key factors: 1) enhanced low-rank recovery by exploiting the graph smoothness assumption, 2) improved graph construction by exploiting clean data recovered by robust PCA. Thus, it boosts the clustering, semi-supervised classification, and data recovery performance overall. Extensive experiments on image/document clustering, object recognition, image shadow removal, and video background subtraction reveal that our model outperforms the previous state-of-the-art methods.
Tasks	graph construction, Image/Document Clustering, Image Shadow Removal, Object Recognition, Video Background Subtraction
Published	2018-12-17
URL	http://arxiv.org/abs/1812.06673v1
PDF	http://arxiv.org/pdf/1812.06673v1.pdf
PWC	https://paperswithcode.com/paper/robust-graph-learning-from-noisy-data
Repo
Framework

Discovering Interesting Plots in Production Yield Data Analytics


Title	Discovering Interesting Plots in Production Yield Data Analytics
Authors	Matthew Nero, Chuanhe Shan, Li-C. Wang, Nik Sumikawa
Abstract	An analytic process is iterative between two agents, an analyst and an analytic toolbox. Each iteration comprises three main steps: preparing a dataset, running an analytic tool, and evaluating the result, where dataset preparation and result evaluation, conducted by the analyst, are largely domain-knowledge driven. In this work, the focus is on automating the result evaluation step. The underlying problem is to identify plots that are deemed interesting by an analyst. We propose a methodology to learn such analyst’s intent based on Generative Adversarial Networks (GANs) and demonstrate its applications in the context of production yield optimization using data collected from several product lines.
Tasks
Published	2018-07-11
URL	http://arxiv.org/abs/1807.03920v1
PDF	http://arxiv.org/pdf/1807.03920v1.pdf
PWC	https://paperswithcode.com/paper/discovering-interesting-plots-in-production
Repo
Framework

SHOMA at Parseme Shared Task on Automatic Identification of VMWEs: Neural Multiword Expression Tagging with High Generalisation


Title	SHOMA at Parseme Shared Task on Automatic Identification of VMWEs: Neural Multiword Expression Tagging with High Generalisation
Authors	Shiva Taslimipoor, Omid Rohanian
Abstract	This paper presents a language-independent deep learning architecture adapted to the task of multiword expression (MWE) identification. We employ a neural architecture comprising of convolutional and recurrent layers with the addition of an optional CRF layer at the top. This system participated in the open track of the Parseme shared task on automatic identification of verbal MWEs due to the use of pre-trained wikipedia word embeddings. It outperformed all participating systems in both open and closed tracks with the overall macro-average MWE-based F1 score of 58.09 averaged among all languages. A particular strength of the system is its superior performance on unseen data entries.
Tasks	Word Embeddings
Published	2018-09-09
URL	http://arxiv.org/abs/1809.03056v1
PDF	http://arxiv.org/pdf/1809.03056v1.pdf
PWC	https://paperswithcode.com/paper/shoma-at-parseme-shared-task-on-automatic
Repo
Framework