October 17, 2019

3218 words 16 mins read

Paper Group ANR 762

Network Reconstruction and Controlling Based on Structural Regularity Analysis. On Training Recurrent Networks with Truncated Backpropagation Through Time in Speech Recognition. Vortex Pooling: Improving Context Representation in Semantic Segmentation. Approximating the covariance ellipsoid. Can LSTM Learn to Capture Agreement? The Case of Basque. …

Network Reconstruction and Controlling Based on Structural Regularity Analysis


Title	Network Reconstruction and Controlling Based on Structural Regularity Analysis
Authors	Tao Wu, Shaojie Qiao, Xingping Xian, Xi-Zhao Wang, Wei Wang, Yanbing Liu
Abstract	From the perspective of network analysis, the ubiquitous networks are comprised of regular and irregular components, which makes uncovering the complexity of network structures to be a fundamental challenge. Exploring the regular information and identifying the roles of microscopic elements in network data can help us recognize the principle of network organization and contribute to network data utilization. However, the intrinsic structural properties of networks remain so far inadequately explored and theorised. With the realistic assumption that there are consistent features across the local structures of networks, we propose a low-rank pursuit based self-representation network model, in which the principle of network organization can be uncovered by a representation matrix. According to this model, original true networks can be reconstructed based on the observed unreliable network topology. In particular, the proposed model enables us to estimate the extent to which the networks are regulable, i.e., measuring the reconstructability of networks. In addition, the model is capable of measuring the importance of microscopic network elements, i.e., nodes and links, in terms of network regularity thereby allowing us to regulate the reconstructability of networks based on them. Extensive experiments on disparate real-world networks demonstrate the effectiveness of the proposed network reconstruction and regulation algorithm. Specifically, the network regularity metric can reflect the reconstructability of networks, and the reconstruction accuracy can be improved by removing irregular network links. Lastly, our approach provides an unique and novel insight into the organization exploring of complex networks.
Tasks
Published	2018-05-20
URL	http://arxiv.org/abs/1805.07746v2
PDF	http://arxiv.org/pdf/1805.07746v2.pdf
PWC	https://paperswithcode.com/paper/network-reconstruction-and-controlling-based
Repo
Framework

On Training Recurrent Networks with Truncated Backpropagation Through Time in Speech Recognition


Title	On Training Recurrent Networks with Truncated Backpropagation Through Time in Speech Recognition
Authors	Hao Tang, James Glass
Abstract	Recurrent neural networks have been the dominant models for many speech and language processing tasks. However, we understand little about the behavior and the class of functions recurrent networks can realize. Moreover, the heuristics used during training complicate the analyses. In this paper, we study recurrent networks’ ability to learn long-term dependency in the context of speech recognition. We consider two decoding approaches, online and batch decoding, and show the classes of functions to which the decoding approaches correspond. We then draw a connection between batch decoding and a popular training approach for recurrent networks, truncated backpropagation through time. Changing the decoding approach restricts the amount of past history recurrent networks can use for prediction, allowing us to analyze their ability to remember. Empirically, we utilize long-term dependency in subphonetic states, phonemes, and words, and show how the design decisions, such as the decoding approach, lookahead, context frames, and consecutive prediction, characterize the behavior of recurrent networks. Finally, we draw a connection between Markov processes and vanishing gradients. These results have implications for studying the long-term dependency in speech data and how these properties are learned by recurrent networks.
Tasks	Speech Recognition
Published	2018-07-09
URL	http://arxiv.org/abs/1807.03396v3
PDF	http://arxiv.org/pdf/1807.03396v3.pdf
PWC	https://paperswithcode.com/paper/on-training-recurrent-networks-with-truncated
Repo
Framework

Vortex Pooling: Improving Context Representation in Semantic Segmentation


Title	Vortex Pooling: Improving Context Representation in Semantic Segmentation
Authors	Chen-Wei Xie, Hong-Yu Zhou, Jianxin Wu
Abstract	Semantic segmentation is a fundamental task in computer vision, which can be considered as a per-pixel classification problem. Recently, although fully convolutional neural network (FCN) based approaches have made remarkable progress in such task, aggregating local and contextual information in convolutional feature maps is still a challenging problem. In this paper, we argue that, when predicting the category of a given pixel, the regions close to the target are more important than those far from it. To tackle this problem, we then propose an effective yet efficient approach named Vortex Pooling to effectively utilize contextual information. Empirical studies are also provided to validate the effectiveness of the proposed method. To be specific, our approach outperforms the previous state-of-the-art model named DeepLab v3 by 1.5% on the PASCAL VOC 2012 val set and 0.6% on the test set by replacing the Atrous Spatial Pyramid Pooling (ASPP) module in DeepLab v3 with the proposed Vortex Pooling. Moreover, our model (10.13FPS) shares similar computation cost with DeepLab v3 (10.37 FPS).
Tasks	Semantic Segmentation
Published	2018-04-17
URL	http://arxiv.org/abs/1804.06242v2
PDF	http://arxiv.org/pdf/1804.06242v2.pdf
PWC	https://paperswithcode.com/paper/vortex-pooling-improving-context
Repo
Framework

Approximating the covariance ellipsoid


Title	Approximating the covariance ellipsoid
Authors	Shahar Mendelson
Abstract	We explore ways in which the covariance ellipsoid ${\cal B}={v \in \mathbb{R}^d : \mathbb{E} <X,v>^2 \leq 1}$ of a centred random vector $X$ in $\mathbb{R}^d$ can be approximated by a simple set. The data one is given for constructing the approximating set consists of $X_1,…,X_N$ that are independent and distributed as $X$. We present a general method that can be used to construct such approximations and implement it for two types of approximating sets. We first construct a (random) set ${\cal K}$ defined by a union of intersections of slabs $H_{z,\alpha}={v \in \mathbb{R}^d : <z,v> \leq \alpha}$ (and therefore ${\cal K}$ is actually the output of a neural network with two hidden layers). The slabs are generated using $X_1,…,X_N$, and under minimal assumptions on $X$ (e.g., $X$ can be heavy-tailed) it suffices that $N = c_1d \eta^{-4}\log(2/\eta)$ to ensure that $(1-\eta) {\cal K} \subset {\cal B} \subset (1+\eta){\cal K}$. In some cases (e.g., if $X$ is rotation invariant and has marginals that are well behaved in some weak sense), a smaller sample size suffices: $N = c_1d\eta^{-2}\log(2/\eta)$. We then show that if the slabs are replaced by randomly generated ellipsoids defined using $X_1,…,X_N$, the same degree of approximation is true when $N \geq c_2d\eta^{-2}\log(2/\eta)$. The construction we use is based on the small-ball method.
Tasks
Published	2018-04-15
URL	http://arxiv.org/abs/1804.05402v1
PDF	http://arxiv.org/pdf/1804.05402v1.pdf
PWC	https://paperswithcode.com/paper/approximating-the-covariance-ellipsoid
Repo
Framework

Can LSTM Learn to Capture Agreement? The Case of Basque


Title	Can LSTM Learn to Capture Agreement? The Case of Basque
Authors	Shauli Ravfogel, Francis M. Tyers, Yoav Goldberg
Abstract	Sequential neural networks models are powerful tools in a variety of Natural Language Processing (NLP) tasks. The sequential nature of these models raises the questions: to what extent can these models implicitly learn hierarchical structures typical to human language, and what kind of grammatical phenomena can they acquire? We focus on the task of agreement prediction in Basque, as a case study for a task that requires implicit understanding of sentence structure and the acquisition of a complex but consistent morphological system. Analyzing experimental results from two syntactic prediction tasks – verb number prediction and suffix recovery – we find that sequential models perform worse on agreement prediction in Basque than one might expect on the basis of a previous agreement prediction work in English. Tentative findings based on diagnostic classifiers suggest the network makes use of local heuristics as a proxy for the hierarchical structure of the sentence. We propose the Basque agreement prediction task as challenging benchmark for models that attempt to learn regularities in human language.
Tasks
Published	2018-09-11
URL	http://arxiv.org/abs/1809.04022v4
PDF	http://arxiv.org/pdf/1809.04022v4.pdf
PWC	https://paperswithcode.com/paper/can-lstm-learn-to-capture-agreement-the-case
Repo
Framework

Efficient learning of smooth probability functions from Bernoulli tests with guarantees


Title	Efficient learning of smooth probability functions from Bernoulli tests with guarantees
Authors	Paul Rolland, Ali Kavis, Alex Immer, Adish Singla, Volkan Cevher
Abstract	We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests. We provide a scalable algorithm for efficiently solving this problem with rigorous guarantees. In particular, we prove the convergence rate of our posterior update rule to the true probability function in L2-norm. Moreover, we allow the Bernoulli tests to depend on contextual features and provide a modified inference engine with provable guarantees for this novel setting. Numerical results show that the empirical convergence rates match the theory, and illustrate the superiority of our approach in handling contextual features over the state-of-the-art.
Tasks
Published	2018-12-11
URL	https://arxiv.org/abs/1812.04428v3
PDF	https://arxiv.org/pdf/1812.04428v3.pdf
PWC	https://paperswithcode.com/paper/efficient-learning-of-smooth-probability
Repo
Framework

Visual Estimation of Building Condition with Patch-level ConvNets


Title	Visual Estimation of Building Condition with Patch-level ConvNets
Authors	David Koch, Miroslav Despotovic, Muntaha Sakeena, Mario Döller, Matthias Zeppelzauer
Abstract	The condition of a building is an important factor for real estate valuation. Currently, the estimation of condition is determined by real estate appraisers which makes it subjective to a certain degree. We propose a novel vision-based approach for the assessment of the building condition from exterior views of the building. To this end, we develop a multi-scale patch-based pattern extraction approach and combine it with convolutional neural networks to estimate building condition from visual clues. Our evaluation shows that visually estimated building condition can serve as a proxy for condition estimates by appraisers.
Tasks
Published	2018-04-26
URL	http://arxiv.org/abs/1804.10113v1
PDF	http://arxiv.org/pdf/1804.10113v1.pdf
PWC	https://paperswithcode.com/paper/visual-estimation-of-building-condition-with
Repo
Framework

Attention-based End-to-End Models for Small-Footprint Keyword Spotting


Title	Attention-based End-to-End Models for Small-Footprint Keyword Spotting
Authors	Changhao Shan, Junbo Zhang, Yujun Wang, Lei Xie
Abstract	In this paper, we propose an attention-based end-to-end neural approach for small-footprint keyword spotting (KWS), which aims to simplify the pipelines of building a production-quality KWS system. Our model consists of an encoder and an attention mechanism. The encoder transforms the input signal into a high level representation using RNNs. Then the attention mechanism weights the encoder features and generates a fixed-length vector. Finally, by linear transformation and softmax function, the vector becomes a score used for keyword detection. We also evaluate the performance of different encoder architectures, including LSTM, GRU and CRNN. Experiments on real-world wake-up data show that our approach outperforms the recent Deep KWS approach by a large margin and the best performance is achieved by CRNN. To be more specific, with ~84K parameters, our attention-based model achieves 1.02% false rejection rate (FRR) at 1.0 false alarm (FA) per hour.
Tasks	Keyword Spotting, Small-Footprint Keyword Spotting
Published	2018-03-29
URL	http://arxiv.org/abs/1803.10916v1
PDF	http://arxiv.org/pdf/1803.10916v1.pdf
PWC	https://paperswithcode.com/paper/attention-based-end-to-end-models-for-small
Repo
Framework

Learning to Fingerprint the Latent Structure in Question Articulation


Title	Learning to Fingerprint the Latent Structure in Question Articulation
Authors	Kumar Mrityunjay, Guntur Ravindra
Abstract	Abstract Machine understanding of questions is tightly related to recognition of articulation in the context of the computational capabilities of an underlying processing algorithm. In this paper a mathematical model to capture and distinguish the latent structure in the articulation of questions is presented. We propose an objective-driven approach to represent this latent structure and show that such an approach is beneficial when examples of complementary objectives are not available. We show that the latent structure can be represented as a system that maximizes a cost function related to the underlying objective. Further, we show that the optimization formulation can be approximated to building a memory of patterns represented as a trained neural auto-encoder. Experimental evaluation using many clusters of questions, each related to an objective, shows 80% recognition accuracy and negligible false positive across these clusters of questions. We then extend the same memory to a related task where the goal is to iteratively refine a dataset of questions based on the latent articulation. We also demonstrate a refinement scheme called K-fingerprints, that achieves nearly 100% recognition with negligible false positive across the different clusters of questions.
Tasks
Published	2018-09-14
URL	http://arxiv.org/abs/1809.05275v1
PDF	http://arxiv.org/pdf/1809.05275v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-fingerprint-the-latent-structure
Repo
Framework

Improving pairwise comparison models using Empirical Bayes shrinkage


Title	Improving pairwise comparison models using Empirical Bayes shrinkage
Authors	Stephen Ragain, Alexander Peysakhovich, Johan Ugander
Abstract	Comparison data arises in many important contexts, e.g. shopping, web clicks, or sports competitions. Typically we are given a dataset of comparisons and wish to train a model to make predictions about the outcome of unseen comparisons. In many cases available datasets have relatively few comparisons (e.g. there are only so many NFL games per year) or efficiency is important (e.g. we want to quickly estimate the relative appeal of a product). In such settings it is well known that shrinkage estimators outperform maximum likelihood estimators. A complicating matter is that standard comparison models such as the conditional multinomial logit model are only models of conditional outcomes (who wins) and not of comparisons themselves (who competes). As such, different models of the comparison process lead to different shrinkage estimators. In this work we derive a collection of methods for estimating the pairwise uncertainty of pairwise predictions based on different assumptions about the comparison process. These uncertainty estimates allow us both to examine model uncertainty as well as perform Empirical Bayes shrinkage estimation of the model parameters. We demonstrate that our shrunk estimators outperform standard maximum likelihood methods on real comparison data from online comparison surveys as well as from several sports contexts.
Tasks
Published	2018-07-24
URL	http://arxiv.org/abs/1807.09236v1
PDF	http://arxiv.org/pdf/1807.09236v1.pdf
PWC	https://paperswithcode.com/paper/improving-pairwise-comparison-models-using
Repo
Framework

Seeing in the dark with recurrent convolutional neural networks


Title	Seeing in the dark with recurrent convolutional neural networks
Authors	Till S. Hartmann
Abstract	Classical convolutional neural networks (cCNNs) are very good at categorizing objects in images. But, unlike human vision which is relatively robust to noise in images, the performance of cCNNs declines quickly as image quality worsens. Here we propose to use recurrent connections within the convolutional layers to make networks robust against pixel noise such as could arise from imaging at low light levels, and thereby significantly increase their performance when tested with simulated noisy video sequences. We show that cCNNs classify images with high signal to noise ratios (SNRs) well, but are easily outperformed when tested with low SNR images (high noise levels) by convolutional neural networks that have recurrency added to convolutional layers, henceforth referred to as gruCNNs. Addition of Bayes-optimal temporal integration to allow the cCNN to integrate multiple image frames still does not match gruCNN performance. Additionally, we show that at low SNRs, the probabilities predicted by the gruCNN (after calibration) have higher confidence than those predicted by the cCNN. We propose to consider recurrent connections in the early stages of neural networks as a solution to computer vision under imperfect lighting conditions and noisy environments; challenges faced during real-time video streams of autonomous driving at night, during rain or snow, and other non-ideal situations.
Tasks	Autonomous Driving, Calibration
Published	2018-11-21
URL	http://arxiv.org/abs/1811.08537v1
PDF	http://arxiv.org/pdf/1811.08537v1.pdf
PWC	https://paperswithcode.com/paper/seeing-in-the-dark-with-recurrent
Repo
Framework

A Hierarchical Neural Network for Sequence-to-Sequences Learning


Title	A Hierarchical Neural Network for Sequence-to-Sequences Learning
Authors	Si Zuo, Zhimin Xu
Abstract	In recent years, the sequence-to-sequence learning neural networks with attention mechanism have achieved great progress. However, there are still challenges, especially for Neural Machine Translation (NMT), such as lower translation quality on long sentences. In this paper, we present a hierarchical deep neural network architecture to improve the quality of long sentences translation. The proposed network embeds sequence-to-sequence neural networks into a two-level category hierarchy by following the coarse-to-fine paradigm. Long sentences are input by splitting them into shorter sequences, which can be well processed by the coarse category network as the long distance dependencies for short sentences is able to be handled by network based on sequence-to-sequence neural network. Then they are concatenated and corrected by the fine category network. The experiments shows that our method can achieve superior results with higher BLEU(Bilingual Evaluation Understudy) scores, lower perplexity and better performance in imitating expression style and words usage than the traditional networks.
Tasks	Machine Translation
Published	2018-11-23
URL	http://arxiv.org/abs/1811.09575v1
PDF	http://arxiv.org/pdf/1811.09575v1.pdf
PWC	https://paperswithcode.com/paper/a-hierarchical-neural-network-for-sequence-to
Repo
Framework

Confidential Boosting with Random Linear Classifiers for Outsourced User-generated Data


Title	Confidential Boosting with Random Linear Classifiers for Outsourced User-generated Data
Authors	Sagar Sharma, Keke Chen
Abstract	User-generated data is crucial to predictive modeling in many applications. With a web/mobile/wearable interface, a data owner can continuously record data generated by distributed users and build various predictive models from the data to improve their operations, services, and revenue. Due to the large size and evolving nature of users data, data owners may rely on public cloud service providers (Cloud) for storage and computation scalability. Exposing sensitive user-generated data and advanced analytic models to Cloud raises privacy concerns. We present a confidential learning framework, SecureBoost, for data owners that want to learn predictive models from aggregated user-generated data but offload the storage and computational burden to Cloud without having to worry about protecting the sensitive data. SecureBoost allows users to submit encrypted or randomly masked data to designated Cloud directly. Our framework utilizes random linear classifiers (RLCs) as the base classifiers in the boosting framework to dramatically simplify the design of the proposed confidential boosting protocols, yet still preserve the model quality. A Cryptographic Service Provider (CSP) is used to assist the Cloud’s processing, reducing the complexity of the protocol constructions. We present two constructions of SecureBoost: HE+GC and SecSh+GC, using combinations of homomorphic encryption, garbled circuits, and random masking to achieve both security and efficiency. For a boosted model, Cloud learns only the RLCs and the CSP learns only the weights of the RLCs. Finally, the data owner collects the two parts to get the complete model. We conduct extensive experiments to understand the quality of the RLC-based boosting and the cost distribution of the constructions. Our results show that SecureBoost can efficiently learn high-quality boosting models from protected user-generated data.
Tasks
Published	2018-02-22
URL	http://arxiv.org/abs/1802.08288v4
PDF	http://arxiv.org/pdf/1802.08288v4.pdf
PWC	https://paperswithcode.com/paper/privacy-preserving-boosting-with-random
Repo
Framework

Nonlinear Dimensionality Reduction on Graphs


Title	Nonlinear Dimensionality Reduction on Graphs
Authors	Yanning Shen, Panagiotis A. Traganitis, Georgios B. Giannakis
Abstract	In this era of data deluge, many signal processing and machine learning tasks are faced with high-dimensional datasets, including images, videos, as well as time series generated from social, commercial and brain network interactions. Their efficient processing calls for dimensionality reduction techniques capable of properly compressing the data while preserving task-related characteristics, going beyond pairwise data correlations. The present paper puts forth a nonlinear dimensionality reduction framework that accounts for data lying on known graphs. The novel framework encompasses most of the existing dimensionality reduction methods, but it is also capable of capturing and preserving possibly nonlinear correlations that are ignored by linear methods. Furthermore, it can take into account information from multiple graphs. The proposed algorithms were tested on synthetic as well as real datasets to corroborate their effectiveness.
Tasks	Dimensionality Reduction, Time Series
Published	2018-01-29
URL	http://arxiv.org/abs/1801.09390v2
PDF	http://arxiv.org/pdf/1801.09390v2.pdf
PWC	https://paperswithcode.com/paper/nonlinear-dimensionality-reduction-on-graphs
Repo
Framework

Adaptive Document Retrieval for Deep Question Answering


Title	Adaptive Document Retrieval for Deep Question Answering
Authors	Bernhard Kratzwald, Stefan Feuerriegel
Abstract	State-of-the-art systems in deep question answering proceed as follows: (1) an initial document retrieval selects relevant documents, which (2) are then processed by a neural network in order to extract the final answer. Yet the exact interplay between both components is poorly understood, especially concerning the number of candidate documents that should be retrieved. We show that choosing a static number of documents – as used in prior research – suffers from a noise-information trade-off and yields suboptimal results. As a remedy, we propose an adaptive document retrieval model. This learns the optimal candidate number for document retrieval, conditional on the size of the corpus and the query. We report extensive experimental results showing that our adaptive approach outperforms state-of-the-art methods on multiple benchmark datasets, as well as in the context of corpora with variable sizes.
Tasks	Question Answering
Published	2018-08-20
URL	http://arxiv.org/abs/1808.06528v1
PDF	http://arxiv.org/pdf/1808.06528v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-document-retrieval-for-deep-question
Repo
Framework