October 18, 2019

3105 words 15 mins read

Paper Group ANR 430

Automatic Generation of Text Descriptive Comments for Code Blocks. Generalized Earley Parser: Bridging Symbolic Grammars and Sequence Data for Future Prediction. Exploring the predictability of range-based volatility estimators using RNNs. End-to-end Speech Recognition with Word-based RNN Language Models. Gaussian Process Conditional Density Estima …

Automatic Generation of Text Descriptive Comments for Code Blocks


Title	Automatic Generation of Text Descriptive Comments for Code Blocks
Authors	Yuding Liang, Kenny Q. Zhu
Abstract	We propose a framework to automatically generate descriptive comments for source code blocks. While this problem has been studied by many researchers previously, their methods are mostly based on fixed template and achieves poor results. Our framework does not rely on any template, but makes use of a new recursive neural network called Code-RNN to extract features from the source code and embed them into one vector. When this vector representation is input to a new recurrent neural network (Code-GRU), the overall framework generates text descriptions of the code with accuracy (Rouge-2 value) significantly higher than other learning-based approaches such as sequence-to-sequence model. The Code-RNN model can also be used in other scenario where the representation of code is required.
Tasks
Published	2018-08-21
URL	http://arxiv.org/abs/1808.06880v1
PDF	http://arxiv.org/pdf/1808.06880v1.pdf
PWC	https://paperswithcode.com/paper/automatic-generation-of-text-descriptive
Repo
Framework

Generalized Earley Parser: Bridging Symbolic Grammars and Sequence Data for Future Prediction


Title	Generalized Earley Parser: Bridging Symbolic Grammars and Sequence Data for Future Prediction
Authors	Siyuan Qi, Baoxiong Jia, Song-Chun Zhu
Abstract	Future predictions on sequence data (e.g., videos or audios) require the algorithms to capture non-Markovian and compositional properties of high-level semantics. Context-free grammars are natural choices to capture such properties, but traditional grammar parsers (e.g., Earley parser) only take symbolic sentences as inputs. In this paper, we generalize the Earley parser to parse sequence data which is neither segmented nor labeled. This generalized Earley parser integrates a grammar parser with a classifier to find the optimal segmentation and labels, and makes top-down future predictions. Experiments show that our method significantly outperforms other approaches for future human activity prediction.
Tasks	Activity Prediction, Future prediction
Published	2018-06-09
URL	http://arxiv.org/abs/1806.03497v1
PDF	http://arxiv.org/pdf/1806.03497v1.pdf
PWC	https://paperswithcode.com/paper/generalized-earley-parser-bridging-symbolic
Repo
Framework

Exploring the predictability of range-based volatility estimators using RNNs


Title	Exploring the predictability of range-based volatility estimators using RNNs
Authors	Gábor Petneházi, József Gáll
Abstract	We investigate the predictability of several range-based stock volatility estimators, and compare them to the standard close-to-close estimator which is most commonly acknowledged as the volatility. The patterns of volatility changes are analyzed using LSTM recurrent neural networks, which are a state of the art method of sequence learning. We implement the analysis on all current constituents of the Dow Jones Industrial Average index, and report averaged evaluation results. We find that changes in the values of range-based estimators are more predictable than that of the estimator using daily closing values only.
Tasks
Published	2018-03-19
URL	http://arxiv.org/abs/1803.07152v1
PDF	http://arxiv.org/pdf/1803.07152v1.pdf
PWC	https://paperswithcode.com/paper/exploring-the-predictability-of-range-based
Repo
Framework

End-to-end Speech Recognition with Word-based RNN Language Models


Title	End-to-end Speech Recognition with Word-based RNN Language Models
Authors	Takaaki Hori, Jaejin Cho, Shinji Watanabe
Abstract	This paper investigates the impact of word-based RNN language models (RNN-LMs) on the performance of end-to-end automatic speech recognition (ASR). In our prior work, we have proposed a multi-level LM, in which character-based and word-based RNN-LMs are combined in hybrid CTC/attention-based ASR. Although this multi-level approach achieves significant error reduction in the Wall Street Journal (WSJ) task, two different LMs need to be trained and used for decoding, which increase the computational cost and memory usage. In this paper, we further propose a novel word-based RNN-LM, which allows us to decode with only the word-based LM, where it provides look-ahead word probabilities to predict next characters instead of the character-based LM, leading competitive accuracy with less computation compared to the multi-level LM. We demonstrate the efficacy of the word-based RNN-LMs using a larger corpus, LibriSpeech, in addition to WSJ we used in the prior work. Furthermore, we show that the proposed model achieves 5.1 %WER for WSJ Eval’92 test set when the vocabulary size is increased, which is the best WER reported for end-to-end ASR systems on this benchmark.
Tasks	End-To-End Speech Recognition, Speech Recognition
Published	2018-08-08
URL	http://arxiv.org/abs/1808.02608v1
PDF	http://arxiv.org/pdf/1808.02608v1.pdf
PWC	https://paperswithcode.com/paper/end-to-end-speech-recognition-with-word-based
Repo
Framework

Gaussian Process Conditional Density Estimation


Title	Gaussian Process Conditional Density Estimation
Authors	Vincent Dutordoir, Hugh Salimbeni, Marc Deisenroth, James Hensman
Abstract	Conditional Density Estimation (CDE) models deal with estimating conditional distributions. The conditions imposed on the distribution are the inputs of the model. CDE is a challenging task as there is a fundamental trade-off between model complexity, representational capacity and overfitting. In this work, we propose to extend the model’s input with latent variables and use Gaussian processes (GP) to map this augmented input onto samples from the conditional distribution. Our Bayesian approach allows for the modeling of small datasets, but we also provide the machinery for it to be applied to big data using stochastic variational inference. Our approach can be used to model densities even in sparse data regions, and allows for sharing learned structure between conditions. We illustrate the effectiveness and wide-reaching applicability of our model on a variety of real-world problems, such as spatio-temporal density estimation of taxi drop-offs, non-Gaussian noise modeling, and few-shot learning on omniglot images.
Tasks	Density Estimation, Few-Shot Learning, Gaussian Processes, Omniglot
Published	2018-10-30
URL	http://arxiv.org/abs/1810.12750v1
PDF	http://arxiv.org/pdf/1810.12750v1.pdf
PWC	https://paperswithcode.com/paper/gaussian-process-conditional-density
Repo
Framework

Top-K Off-Policy Correction for a REINFORCE Recommender System


Title	Top-K Off-Policy Correction for a REINFORCE Recommender System
Authors	Minmin Chen, Alex Beutel, Paul Covington, Sagar Jain, Francois Belletti, Ed Chi
Abstract	Industrial recommender systems deal with extremely large action spaces – many millions of items to recommend. Moreover, they need to serve billions of users, who are unique at any point in time, making a complex user state space. Luckily, huge quantities of logged implicit feedback (e.g., user clicks, dwell time) are available for learning. Learning from the logged feedback is however subject to biases caused by only observing feedback on recommendations selected by the previous versions of the recommender. In this work, we present a general recipe of addressing such biases in a production top-K recommender system at Youtube, built with a policy-gradient-based algorithm, i.e. REINFORCE. The contributions of the paper are: (1) scaling REINFORCE to a production recommender system with an action space on the orders of millions; (2) applying off-policy correction to address data biases in learning from logged feedback collected from multiple behavior policies; (3) proposing a novel top-K off-policy correction to account for our policy recommending multiple items at a time; (4) showcasing the value of exploration. We demonstrate the efficacy of our approaches through a series of simulations and multiple live experiments on Youtube.
Tasks	Recommendation Systems
Published	2018-12-06
URL	http://arxiv.org/abs/1812.02353v1
PDF	http://arxiv.org/pdf/1812.02353v1.pdf
PWC	https://paperswithcode.com/paper/top-k-off-policy-correction-for-a-reinforce
Repo
Framework

State Classification with CNN


Title	State Classification with CNN
Authors	Astha Sharma
Abstract	There is a plenty of research going on in field of object recognition, but object state recognition has not been addressed as much. There are many important applications which can utilize object state recognition, such as, in robotics, to decide for how to grab an object. A convolution neural network was designed to classify an image to one of its states. The approach used for training is transfer learning with Inception v3 module of GoogLeNet used as the pre-trained model. The model was trained on images of 18 cooking objects and tested on another set of cooking objects. The model was able to classify those images with 76% accuracy.
Tasks	Object Recognition, Transfer Learning
Published	2018-06-05
URL	http://arxiv.org/abs/1806.03973v2
PDF	http://arxiv.org/pdf/1806.03973v2.pdf
PWC	https://paperswithcode.com/paper/state-classification-with-cnn
Repo
Framework

Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training


Title	Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training
Authors	Youjie Li, Mingchao Yu, Songze Li, Salman Avestimehr, Nam Sung Kim, Alexander Schwing
Abstract	Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands. Classical distributed approaches, synchronous or asynchronous, are based on the parameter server architecture, i.e., worker nodes compute gradients which are communicated to the parameter server while updated parameters are returned. Recently, distributed training with AllReduce operations gained popularity as well. While many of those operations seem appealing, little is reported about wall-clock training time improvements. In this paper, we carefully analyze the AllReduce based setup, propose timing models which include network latency, bandwidth, cluster size and compute time, and demonstrate that a pipelined training with a width of two combines the best of both synchronous and asynchronous training. Specifically, for a setup consisting of a four-node GPU cluster we show wall-clock time training improvements of up to 5.4x compared to conventional approaches.
Tasks
Published	2018-11-08
URL	http://arxiv.org/abs/1811.03619v3
PDF	http://arxiv.org/pdf/1811.03619v3.pdf
PWC	https://paperswithcode.com/paper/pipe-sgd-a-decentralized-pipelined-sgd
Repo
Framework

Empirical Explorations in Training Networks with Discrete Activations


Title	Empirical Explorations in Training Networks with Discrete Activations
Authors	Shumeet Baluja
Abstract	We present extensive experiments training and testing hidden units in deep networks that emit only a predefined, static, number of discretized values. These units provide benefits in real-world deployment in systems in which memory and/or computation may be limited. Additionally, they are particularly well suited for use in large recurrent network models that require the maintenance of large amounts of internal state in memory. Surprisingly, we find that despite reducing the number of values that can be represented in the output activations from $2^{32}-2^{64}$ to between 64 and 256, there is little to no degradation in network performance across a variety of different settings. We investigate simple classification and regression tasks, as well as memorization and compression problems. We compare the results with more standard activations, such as tanh and relu. Unlike previous discretization studies which often concentrate only on binary units, we examine the effects of varying the number of allowed activation levels. Compared to existing approaches for discretization, the approach presented here is both conceptually and programatically simple, has no stochastic component, and allows the training, testing, and usage phases to be treated in exactly the same manner.
Tasks
Published	2018-01-16
URL	http://arxiv.org/abs/1801.05156v1
PDF	http://arxiv.org/pdf/1801.05156v1.pdf
PWC	https://paperswithcode.com/paper/empirical-explorations-in-training-networks
Repo
Framework

MILD-Net: Minimal Information Loss Dilated Network for Gland Instance Segmentation in Colon Histology Images


Title	MILD-Net: Minimal Information Loss Dilated Network for Gland Instance Segmentation in Colon Histology Images
Authors	Simon Graham, Hao Chen, Jevgenij Gamper, Qi Dou, Pheng-Ann Heng, David Snead, Yee Wah Tsang, Nasir Rajpoot
Abstract	The analysis of glandular morphology within colon histopathology images is an important step in determining the grade of colon cancer. Despite the importance of this task, manual segmentation is laborious, time-consuming and can suffer from subjectivity among pathologists. The rise of computational pathology has led to the development of automated methods for gland segmentation that aim to overcome the challenges of manual segmentation. However, this task is non-trivial due to the large variability in glandular appearance and the difficulty in differentiating between certain glandular and non-glandular histological structures. Furthermore, a measure of uncertainty is essential for diagnostic decision making. To address these challenges, we propose a fully convolutional neural network that counters the loss of information caused by max-pooling by re-introducing the original image at multiple points within the network. We also use atrous spatial pyramid pooling with varying dilation rates for preserving the resolution and multi-level aggregation. To incorporate uncertainty, we introduce random transformations during test time for an enhanced segmentation result that simultaneously generates an uncertainty map, highlighting areas of ambiguity. We show that this map can be used to define a metric for disregarding predictions with high uncertainty. The proposed network achieves state-of-the-art performance on the GlaS challenge dataset and on a second independent colorectal adenocarcinoma dataset. In addition, we perform gland instance segmentation on whole-slide images from two further datasets to highlight the generalisability of our method. As an extension, we introduce MILD-Net+ for simultaneous gland and lumen segmentation, to increase the diagnostic power of the network.
Tasks	Decision Making, Instance Segmentation, Semantic Segmentation
Published	2018-06-05
URL	http://arxiv.org/abs/1806.01963v4
PDF	http://arxiv.org/pdf/1806.01963v4.pdf
PWC	https://paperswithcode.com/paper/mild-net-minimal-information-loss-dilated
Repo
Framework

Privacy Preservation in Location-Based Services: A Novel Metric and Attack Model


Title	Privacy Preservation in Location-Based Services: A Novel Metric and Attack Model
Authors	Sina Shaham, Ming Ding, Bo Liu, Zihuai Lin, Jun Li
Abstract	Recent years have seen rising needs for location-based services in our everyday life. Aside from the many advantages provided by these services, they have caused serious concerns regarding the location privacy of users. An adversary such as an untrusted location-based server can monitor the queried locations by a user to infer critical information such as the user’s home address, health conditions, shopping habits, etc. To address this issue, dummy-based algorithms have been developed to increase the anonymity of users, and thus, protecting their privacy. Unfortunately, the existing algorithms only consider a limited amount of side information known by an adversary which may face more serious challenges in practice. In this paper, we incorporate a new type of side information based on consecutive location changes of users and propose a new metric called transition-entropy to investigate the location privacy preservation, followed by two algorithms to improve the transition-entropy for a given dummy generation algorithm. Then, we develop an attack model based on the Viterbi algorithm which can significantly threaten the location privacy of the users. Next, in order to protect the users from Viterbi attack, we propose an algorithm called robust dummy generation (RDG) which can resist against the Viterbi attack while maintaining a high performance in terms of the privacy metrics introduced in the paper. All the algorithms are applied and analyzed on a real-life dataset.
Tasks
Published	2018-05-16
URL	http://arxiv.org/abs/1805.06104v1
PDF	http://arxiv.org/pdf/1805.06104v1.pdf
PWC	https://paperswithcode.com/paper/privacy-preservation-in-location-based
Repo
Framework

An Approximation Algorithm for Risk-averse Submodular Optimization


Title	An Approximation Algorithm for Risk-averse Submodular Optimization
Authors	Lifeng Zhou, Pratap Tokekar
Abstract	We study the problem of incorporating risk while making combinatorial decisions under uncertainty. We formulate a discrete submodular maximization problem for selecting a set using Conditional-Value-at-Risk (CVaR), a risk metric commonly used in financial analysis. While CVaR has recently been used in optimization of linear cost functions in robotics, we take the first stages towards extending this to discrete submodular optimization and provide several positive results. Specifically, we propose the Sequential Greedy Algorithm that provides an approximation guarantee on finding the maxima of the CVaR cost function under a matroidal constraint. The approximation guarantee shows that the solution produced by our algorithm is within a constant factor of the optimal and an additive term that depends on the optimal. Our analysis uses the curvature of the submodular set function, and proves that the algorithm runs in polynomial time. This formulates a number of combinatorial optimization problems that appear in robotics. We use two such problems, vehicle assignment under uncertainty for mobility-on-demand and sensor selection with failures for environmental monitoring, as case studies to demonstrate the efficacy of our formulation.
Tasks	Combinatorial Optimization
Published	2018-07-24
URL	http://arxiv.org/abs/1807.09358v2
PDF	http://arxiv.org/pdf/1807.09358v2.pdf
PWC	https://paperswithcode.com/paper/an-approximation-algorithm-for-risk-averse
Repo
Framework

RUSSE: The First Workshop on Russian Semantic Similarity


Title	RUSSE: The First Workshop on Russian Semantic Similarity
Authors	Alexander Panchenko, Natalia Loukachevitch, Dmitry Ustalov, Denis Paperno, Christian Meyer, Natalia Konstantinova
Abstract	The paper gives an overview of the Russian Semantic Similarity Evaluation (RUSSE) shared task held in conjunction with the Dialogue 2015 conference. There exist a lot of comparative studies on semantic similarity, yet no analysis of such measures was ever performed for the Russian language. Exploring this problem for the Russian language is even more interesting, because this language has features, such as rich morphology and free word order, which make it significantly different from English, German, and other well-studied languages. We attempt to bridge this gap by proposing a shared task on the semantic similarity of Russian nouns. Our key contribution is an evaluation methodology based on four novel benchmark datasets for the Russian language. Our analysis of the 105 submissions from 19 teams reveals that successful approaches for English, such as distributional and skip-gram models, are directly applicable to Russian as well. On the one hand, the best results in the contest were obtained by sophisticated supervised models that combine evidence from different sources. On the other hand, completely unsupervised approaches, such as a skip-gram model estimated on a large-scale corpus, were able score among the top 5 systems.
Tasks	Semantic Similarity, Semantic Textual Similarity
Published	2018-03-15
URL	http://arxiv.org/abs/1803.05820v1
PDF	http://arxiv.org/pdf/1803.05820v1.pdf
PWC	https://paperswithcode.com/paper/russe-the-first-workshop-on-russian-semantic
Repo
Framework

Capturing Between-Tasks Covariance and Similarities Using Multivariate Linear Mixed Models


Title	Capturing Between-Tasks Covariance and Similarities Using Multivariate Linear Mixed Models
Authors	Aviv Navon, Saharon Rosset
Abstract	We consider the problem of predicting several response variables using the same set of explanatory variables. This setting naturally induces a group structure over the coefficient matrix, in which every explanatory variable corresponds to a set of related coefficients. Most of the existing methods that utilize this group formation assume that the similarities between related coefficients arise solely through a joint sparsity structure. In this paper, we propose a procedure for constructing an estimator of a multivariate regression coefficient matrix that directly models and captures the within-group similarities, by employing a multivariate linear mixed model formulation, with a joint estimation of covariance matrices for coefficients and errors via penalized likelihood. Our approach, which we term Multivariate random Regression with Covariance Estimation (MrRCE) encourages structured similarity in parameters, in which coefficients for the same variable in related tasks sharing the same sign and similar magnitude. We illustrate the benefits of our approach in synthetic and real examples, and show that the proposed method outperforms natural competitors and alternative estimators under several model settings.
Tasks
Published	2018-12-10
URL	https://arxiv.org/abs/1812.03662v2
PDF	https://arxiv.org/pdf/1812.03662v2.pdf
PWC	https://paperswithcode.com/paper/capturing-between-tasks-covariance-and
Repo
Framework

Ratio-Preserving Half-Cylindrical Warps for Natural Image Stitching


Title	Ratio-Preserving Half-Cylindrical Warps for Natural Image Stitching
Authors	Yifang Xu, Jing Chen, Tianli Liao
Abstract	A novel warp for natural image stitching is proposed that utilizes the property of cylindrical warp and a horizontal pixel selection strategy. The proposed ratio-preserving half-cylindrical warp is a combination of homography and cylindrical warps which guarantees alignment by homography and possesses less projective distortion by cylindrical warp. Unlike previous approaches applying cylindrical warp before homography, we use partition lines to divide the image into different parts and apply homography in the overlapping region while a composition of homography and cylindrical warps in the non-overlapping region. The pixel selection strategy then samples the points in horizontal and reconstructs the image via interpolation to further reduce horizontal distortion by maintaining the ratio as similarity. With applying half-cylindrical warp and horizontal pixel selection, the projective distortion in vertical and horizontal is mitigated simultaneously. Experiments show that our warp is efficient and produces a more natural-looking stitched result than previous methods.
Tasks	Image Stitching
Published	2018-03-18
URL	http://arxiv.org/abs/1803.06655v1
PDF	http://arxiv.org/pdf/1803.06655v1.pdf
PWC	https://paperswithcode.com/paper/ratio-preserving-half-cylindrical-warps-for
Repo
Framework