May 5, 2019

2930 words 14 mins read

Paper Group ANR 446

Neural Name Translation Improves Neural Machine Translation. Recurrent Neural Network Encoder with Attention for Community Question Answering. Probabilistic structure discovery in time series data. Deep Neural Networks to Enable Real-time Multimessenger Astrophysics. Self-Transfer Learning for Fully Weakly Supervised Object Localization. Geometry-a …

Neural Name Translation Improves Neural Machine Translation


Title	Neural Name Translation Improves Neural Machine Translation
Authors	Xiaoqing Li, Jiajun Zhang, Chengqing Zong
Abstract	In order to control computational complexity, neural machine translation (NMT) systems convert all rare words outside the vocabulary into a single unk symbol. Previous solution (Luong et al., 2015) resorts to use multiple numbered unks to learn the correspondence between source and target rare words. However, testing words unseen in the training corpus cannot be handled by this method. And it also suffers from the noisy word alignment. In this paper, we focus on a major type of rare words – named entity (NE), and propose to translate them with character level sequence to sequence model. The NE translation model is further used to derive high quality NE alignment in the bilingual training corpus. With the integration of NE translation and alignment modules, our NMT system is able to surpass the baseline system by 2.9 BLEU points on the Chinese to English task.
Tasks	Machine Translation, Word Alignment
Published	2016-07-07
URL	http://arxiv.org/abs/1607.01856v1
PDF	http://arxiv.org/pdf/1607.01856v1.pdf
PWC	https://paperswithcode.com/paper/neural-name-translation-improves-neural
Repo
Framework

Recurrent Neural Network Encoder with Attention for Community Question Answering


Title	Recurrent Neural Network Encoder with Attention for Community Question Answering
Authors	Wei-Ning Hsu, Yu Zhang, James Glass
Abstract	We apply a general recurrent neural network (RNN) encoder framework to community question answering (cQA) tasks. Our approach does not rely on any linguistic processing, and can be applied to different languages or domains. Further improvements are observed when we extend the RNN encoders with a neural attention mechanism that encourages reasoning over entire sequences. To deal with practical issues such as data sparsity and imbalanced labels, we apply various techniques such as transfer learning and multitask learning. Our experiments on the SemEval-2016 cQA task show 10% improvement on a MAP score compared to an information retrieval-based approach, and achieve comparable performance to a strong handcrafted feature-based method.
Tasks	Community Question Answering, Information Retrieval, Question Answering, Transfer Learning
Published	2016-03-23
URL	http://arxiv.org/abs/1603.07044v1
PDF	http://arxiv.org/pdf/1603.07044v1.pdf
PWC	https://paperswithcode.com/paper/recurrent-neural-network-encoder-with
Repo
Framework

Probabilistic structure discovery in time series data


Title	Probabilistic structure discovery in time series data
Authors	David Janz, Brooks Paige, Tom Rainforth, Jan-Willem van de Meent, Frank Wood
Abstract	Existing methods for structure discovery in time series data construct interpretable, compositional kernels for Gaussian process regression models. While the learned Gaussian process model provides posterior mean and variance estimates, typically the structure is learned via a greedy optimization procedure. This restricts the space of possible solutions and leads to over-confident uncertainty estimates. We introduce a fully Bayesian approach, inferring a full posterior over structures, which more reliably captures the uncertainty of the model.
Tasks	Time Series
Published	2016-11-21
URL	http://arxiv.org/abs/1611.06863v1
PDF	http://arxiv.org/pdf/1611.06863v1.pdf
PWC	https://paperswithcode.com/paper/probabilistic-structure-discovery-in-time
Repo
Framework

Deep Neural Networks to Enable Real-time Multimessenger Astrophysics


Title	Deep Neural Networks to Enable Real-time Multimessenger Astrophysics
Authors	Daniel George, E. A. Huerta
Abstract	Gravitational wave astronomy has set in motion a scientific revolution. To further enhance the science reach of this emergent field, there is a pressing need to increase the depth and speed of the gravitational wave algorithms that have enabled these groundbreaking discoveries. To contribute to this effort, we introduce Deep Filtering, a new highly scalable method for end-to-end time-series signal processing, based on a system of two deep convolutional neural networks, which we designed for classification and regression to rapidly detect and estimate parameters of signals in highly noisy time-series data streams. We demonstrate a novel training scheme with gradually increasing noise levels, and a transfer learning procedure between the two networks. We showcase the application of this method for the detection and parameter estimation of gravitational waves from binary black hole mergers. Our results indicate that Deep Filtering significantly outperforms conventional machine learning techniques, achieves similar performance compared to matched-filtering while being several orders of magnitude faster thus allowing real-time processing of raw big data with minimal resources. More importantly, Deep Filtering extends the range of gravitational wave signals that can be detected with ground-based gravitational wave detectors. This framework leverages recent advances in artificial intelligence algorithms and emerging hardware architectures, such as deep-learning-optimized GPUs, to facilitate real-time searches of gravitational wave sources and their electromagnetic and astro-particle counterparts.
Tasks	Time Series, Transfer Learning
Published	2016-12-30
URL	http://arxiv.org/abs/1701.00008v3
PDF	http://arxiv.org/pdf/1701.00008v3.pdf
PWC	https://paperswithcode.com/paper/deep-neural-networks-to-enable-real-time
Repo
Framework

Self-Transfer Learning for Fully Weakly Supervised Object Localization


Title	Self-Transfer Learning for Fully Weakly Supervised Object Localization
Authors	Sangheum Hwang, Hyo-Eun Kim
Abstract	Recent advances of deep learning have achieved remarkable performances in various challenging computer vision tasks. Especially in object localization, deep convolutional neural networks outperform traditional approaches based on extraction of data/task-driven features instead of hand-crafted features. Although location information of region-of-interests (ROIs) gives good prior for object localization, it requires heavy annotation efforts from human resources. Thus a weakly supervised framework for object localization is introduced. The term “weakly” means that this framework only uses image-level labeled datasets to train a network. With the help of transfer learning which adopts weight parameters of a pre-trained network, the weakly supervised learning framework for object localization performs well because the pre-trained network already has well-trained class-specific features. However, those approaches cannot be used for some applications which do not have pre-trained networks or well-localized large scale images. Medical image analysis is a representative among those applications because it is impossible to obtain such pre-trained networks. In this work, we present a “fully” weakly supervised framework for object localization (“semi”-weakly is the counterpart which uses pre-trained filters for weakly supervised localization) named as self-transfer learning (STL). It jointly optimizes both classification and localization networks simultaneously. By controlling a supervision level of the localization network, STL helps the localization network focus on correct ROIs without any types of priors. We evaluate the proposed STL framework using two medical image datasets, chest X-rays and mammograms, and achieve signiticantly better localization performance compared to previous weakly supervised approaches.
Tasks	Object Localization, Transfer Learning, Weakly-Supervised Object Localization
Published	2016-02-04
URL	http://arxiv.org/abs/1602.01625v1
PDF	http://arxiv.org/pdf/1602.01625v1.pdf
PWC	https://paperswithcode.com/paper/self-transfer-learning-for-fully-weakly
Repo
Framework

Geometry-aware Similarity Learning on SPD Manifolds for Visual Recognition


Title	Geometry-aware Similarity Learning on SPD Manifolds for Visual Recognition
Authors	Zhiwu Huang, Ruiping Wang, Xianqiu Li, Wenxian Liu, Shiguang Shan, Luc Van Gool, Xilin Chen
Abstract	Symmetric Positive Definite (SPD) matrices have been widely used for data representation in many visual recognition tasks. The success mainly attributes to learning discriminative SPD matrices with encoding the Riemannian geometry of the underlying SPD manifold. In this paper, we propose a geometry-aware SPD similarity learning (SPDSL) framework to learn discriminative SPD features by directly pursuing manifold-manifold transformation matrix of column full-rank. Specifically, by exploiting the Riemannian geometry of the manifold of fixed-rank Positive Semidefinite (PSD) matrices, we present a new solution to reduce optimizing over the space of column full-rank transformation matrices to optimizing on the PSD manifold which has a well-established Riemannian structure. Under this solution, we exploit a new supervised SPD similarity learning technique to learn the transformation by regressing the similarities of selected SPD data pairs to their ground-truth similarities on the target SPD manifold. To optimize the proposed objective function, we further derive an algorithm on the PSD manifold. Evaluations on three visual classification tasks show the advantages of the proposed approach over the existing SPD-based discriminant learning methods.
Tasks
Published	2016-08-17
URL	http://arxiv.org/abs/1608.04914v1
PDF	http://arxiv.org/pdf/1608.04914v1.pdf
PWC	https://paperswithcode.com/paper/geometry-aware-similarity-learning-on-spd
Repo
Framework

Tumour ROI Estimation in Ultrasound Images via Radon Barcodes in Patients with Locally Advanced Breast Cancer


Title	Tumour ROI Estimation in Ultrasound Images via Radon Barcodes in Patients with Locally Advanced Breast Cancer
Authors	Hamid R. Tizhoosh, Mehrdad J. Gangeh, Hadi Tadayyon, Gregory J. Czarnota
Abstract	Quantitative ultrasound (QUS) methods provide a promising framework that can non-invasively and inexpensively be used to predict or assess the tumour response to cancer treatment. The first step in using the QUS methods is to select a region of interest (ROI) inside the tumour in ultrasound images. Manual segmentation, however, is very time consuming and tedious. In this paper, a semi-automated approach will be proposed to roughly localize an ROI for a tumour in ultrasound images of patients with locally advanced breast cancer (LABC). Content-based barcodes, a recently introduced binary descriptor based on Radon transform, were used in order to find similar cases and estimate a bounding box surrounding the tumour. Experiments with 33 B-scan images resulted in promising results with an accuracy of $81%$.
Tasks
Published	2016-02-08
URL	http://arxiv.org/abs/1602.02586v1
PDF	http://arxiv.org/pdf/1602.02586v1.pdf
PWC	https://paperswithcode.com/paper/tumour-roi-estimation-in-ultrasound-images
Repo
Framework

Shape from Mixed Polarization


Title	Shape from Mixed Polarization
Authors	Vage Taamazyan, Achuta Kadambi, Ramesh Raskar
Abstract	Shape from Polarization (SfP) estimates surface normals using photos captured at different polarizer rotations. Fundamentally, the SfP model assumes that light is reflected either diffusely or specularly. However, this model is not valid for many real-world surfaces exhibiting a mixture of diffuse and specular properties. To address this challenge, previous methods have used a sequential solution: first, use an existing algorithm to separate the scene into diffuse and specular components, then apply the appropriate SfP model. In this paper, we propose a new method that jointly uses viewpoint and polarization data to holistically separate diffuse and specular components, recover refractive index, and ultimately recover 3D shape. By involving the physics of polarization in the separation process, we demonstrate competitive results with a benchmark method, while recovering additional information (e.g. refractive index).
Tasks
Published	2016-05-06
URL	http://arxiv.org/abs/1605.02066v2
PDF	http://arxiv.org/pdf/1605.02066v2.pdf
PWC	https://paperswithcode.com/paper/shape-from-mixed-polarization
Repo
Framework

Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation


Title	Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation
Authors	Wojciech Samek, Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Klaus-Robert Müller
Abstract	Complex nonlinear models such as deep neural network (DNNs) have become an important tool for image classification, speech recognition, natural language processing, and many other fields of application. These models however lack transparency due to their complex nonlinear structure and to the complex data distributions to which they typically apply. As a result, it is difficult to fully characterize what makes these models reach a particular decision for a given input. This lack of transparency can be a drawback, especially in the context of sensitive applications such as medical analysis or security. In this short paper, we summarize a recent technique introduced by Bach et al. [1] that explains predictions by decomposing the classification decision of DNN models in terms of input variables.
Tasks	Image Classification, Speech Recognition
Published	2016-11-24
URL	http://arxiv.org/abs/1611.08191v1
PDF	http://arxiv.org/pdf/1611.08191v1.pdf
PWC	https://paperswithcode.com/paper/interpreting-the-predictions-of-complex-ml
Repo
Framework

End-to-End Pedestrian Collision Warning System based on a Convolutional Neural Network with Semantic Segmentation


Title	End-to-End Pedestrian Collision Warning System based on a Convolutional Neural Network with Semantic Segmentation
Authors	Heechul Jung, Min-Kook Choi, Kwon Soon, Woo Young Jung
Abstract	Traditional pedestrian collision warning systems sometimes raise alarms even when there is no danger (e.g., when all pedestrians are walking on the sidewalk). These false alarms can make it difficult for drivers to concentrate on their driving. In this paper, we propose a novel framework for an end-to-end pedestrian collision warning system based on a convolutional neural network. Semantic segmentation information is used to train the convolutional neural network and two loss functions, such as cross entropy and Euclidean losses, are minimized. Finally, we demonstrate the effectiveness of our method in reducing false alarms and increasing warning accuracy compared to a traditional histogram of oriented gradients (HoG)-based system.
Tasks	Semantic Segmentation
Published	2016-12-20
URL	http://arxiv.org/abs/1612.06558v1
PDF	http://arxiv.org/pdf/1612.06558v1.pdf
PWC	https://paperswithcode.com/paper/end-to-end-pedestrian-collision-warning
Repo
Framework

Combining observational and experimental data to find heterogeneous treatment effects


Title	Combining observational and experimental data to find heterogeneous treatment effects
Authors	Alexander Peysakhovich, Akos Lada
Abstract	Every design choice will have different effects on different units. However traditional A/B tests are often underpowered to identify these heterogeneous effects. This is especially true when the set of unit-level attributes is high-dimensional and our priors are weak about which particular covariates are important. However, there are often observational data sets available that are orders of magnitude larger. We propose a method to combine these two data sources to estimate heterogeneous treatment effects. First, we use observational time series data to estimate a mapping from covariates to unit-level effects. These estimates are likely biased but under some conditions the bias preserves unit-level relative rank orderings. If these conditions hold, we only need sufficient experimental data to identify a monotonic, one-dimensional transformation from observationally predicted treatment effects to real treatment effects. This reduces power demands greatly and makes the detection of heterogeneous effects much easier. As an application, we show how our method can be used to improve Facebook page recommendations.
Tasks	Time Series
Published	2016-11-08
URL	http://arxiv.org/abs/1611.02385v1
PDF	http://arxiv.org/pdf/1611.02385v1.pdf
PWC	https://paperswithcode.com/paper/combining-observational-and-experimental-data
Repo
Framework

Compact Deep Convolutional Neural Networks With Coarse Pruning


Title	Compact Deep Convolutional Neural Networks With Coarse Pruning
Authors	Sajid Anwar, Wonyong Sung
Abstract	The learning capability of a neural network improves with increasing depth at higher computational costs. Wider layers with dense kernel connectivity patterns furhter increase this cost and may hinder real-time inference. We propose feature map and kernel level pruning for reducing the computational complexity of a deep convolutional neural network. Pruning feature maps reduces the width of a layer and hence does not need any sparse representation. Further, kernel pruning converts the dense connectivity pattern into a sparse one. Due to coarse nature, these pruning granularities can be exploited by GPUs and VLSI based implementations. We propose a simple and generic strategy to choose the least adversarial pruning masks for both granularities. The pruned networks are retrained which compensates the loss in accuracy. We obtain the best pruning ratios when we prune a network with both granularities. Experiments with the CIFAR-10 dataset show that more than 85% sparsity can be induced in the convolution layers with less than 1% increase in the missclassification rate of the baseline network.
Tasks	Network Pruning
Published	2016-10-30
URL	http://arxiv.org/abs/1610.09639v1
PDF	http://arxiv.org/pdf/1610.09639v1.pdf
PWC	https://paperswithcode.com/paper/compact-deep-convolutional-neural-networks
Repo
Framework

Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation


Title	Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
Authors	Jose Caballero, Christian Ledig, Andrew Aitken, Alejandro Acosta, Johannes Totz, Zehan Wang, Wenzhe Shi
Abstract	Convolutional neural networks have enabled accurate image super-resolution in real-time. However, recent attempts to benefit from temporal correlations in video super-resolution have been limited to naive or inefficient architectures. In this paper, we introduce spatio-temporal sub-pixel convolution networks that effectively exploit temporal redundancies and improve reconstruction accuracy while maintaining real-time speed. Specifically, we discuss the use of early fusion, slow fusion and 3D convolutions for the joint processing of multiple consecutive video frames. We also propose a novel joint motion compensation and video super-resolution algorithm that is orders of magnitude more efficient than competing methods, relying on a fast multi-resolution spatial transformer module that is end-to-end trainable. These contributions provide both higher accuracy and temporally more consistent videos, which we confirm qualitatively and quantitatively. Relative to single-frame models, spatio-temporal networks can either reduce the computational cost by 30% whilst maintaining the same quality or provide a 0.2dB gain for a similar computational cost. Results on publicly available datasets demonstrate that the proposed algorithms surpass current state-of-the-art performance in both accuracy and efficiency.
Tasks	Motion Compensation, Video Super-Resolution
Published	2016-11-16
URL	http://arxiv.org/abs/1611.05250v2
PDF	http://arxiv.org/pdf/1611.05250v2.pdf
PWC	https://paperswithcode.com/paper/real-time-video-super-resolution-with-spatio
Repo
Framework

Automatic Attribute Discovery with Neural Activations


Title	Automatic Attribute Discovery with Neural Activations
Authors	Sirion Vittayakorn, Takayuki Umeda, Kazuhiko Murasaki, Kyoko Sudo, Takayuki Okatani, Kota Yamaguchi
Abstract	How can a machine learn to recognize visual attributes emerging out of online community without a definitive supervised dataset? This paper proposes an automatic approach to discover and analyze visual attributes from a noisy collection of image-text data on the Web. Our approach is based on the relationship between attributes and neural activations in the deep network. We characterize the visual property of the attribute word as a divergence within weakly-annotated set of images. We show that the neural activations are useful for discovering and learning a classifier that well agrees with human perception from the noisy real-world Web data. The empirical study suggests the layered structure of the deep neural networks also gives us insights into the perceptual depth of the given word. Finally, we demonstrate that we can utilize highly-activating neurons for finding semantically relevant regions.
Tasks
Published	2016-07-25
URL	http://arxiv.org/abs/1607.07262v1
PDF	http://arxiv.org/pdf/1607.07262v1.pdf
PWC	https://paperswithcode.com/paper/automatic-attribute-discovery-with-neural
Repo
Framework

Generating Sentiment Lexicons for German Twitter


Title	Generating Sentiment Lexicons for German Twitter
Authors	Uladzimir Sidarenka, Manfred Stede
Abstract	Despite a substantial progress made in developing new sentiment lexicon generation (SLG) methods for English, the task of transferring these approaches to other languages and domains in a sound way still remains open. In this paper, we contribute to the solution of this problem by systematically comparing semi-automatic translations of common English polarity lists with the results of the original automatic SLG algorithms, which were applied directly to German data. We evaluate these lexicons on a corpus of 7,992 manually annotated tweets. In addition to that, we also collate the results of dictionary- and corpus-based SLG methods in order to find out which of these paradigms is better suited for the inherently noisy domain of social media. Our experiments show that semi-automatic translations notably outperform automatic systems (reaching a macro-averaged F1-score of 0.589), and that dictionary-based techniques produce much better polarity lists as compared to corpus-based approaches (whose best F1-scores run up to 0.479 and 0.419 respectively) even for the non-standard Twitter genre.
Tasks
Published	2016-10-31
URL	http://arxiv.org/abs/1610.09995v1
PDF	http://arxiv.org/pdf/1610.09995v1.pdf
PWC	https://paperswithcode.com/paper/generating-sentiment-lexicons-for-german
Repo
Framework