October 16, 2019

3044 words 15 mins read

Paper Group ANR 1071

Deep Learning for Audio Transcription on Low-Resource Datasets. Story Disambiguation: Tracking Evolving News Stories across News and Social Streams. Analysis of Langevin Monte Carlo via convex optimization. Super-Resolution via Conditional Implicit Maximum Likelihood Estimation. Counting to Explore and Generalize in Text-based Games. Hierarchical c …

Deep Learning for Audio Transcription on Low-Resource Datasets


Title	Deep Learning for Audio Transcription on Low-Resource Datasets
Authors	Veronica Morfi, Dan Stowell
Abstract	In training a deep learning system to perform audio transcription, two practical problems may arise. Firstly, most datasets are weakly labelled, having only a list of events present in each recording without any temporal information for training. Secondly, deep neural networks need a very large amount of labelled training data to achieve good quality performance, yet in practice it is difficult to collect enough samples for most classes of interest. In this paper, we propose factorising the final task of audio transcription into multiple intermediate tasks in order to improve the training performance when dealing with this kind of low-resource datasets. We evaluate three data-efficient approaches of training a stacked convolutional and recurrent neural network for the intermediate tasks. Our results show that different methods of training have different advantages and disadvantages.
Tasks
Published	2018-07-10
URL	http://arxiv.org/abs/1807.03697v2
PDF	http://arxiv.org/pdf/1807.03697v2.pdf
PWC	https://paperswithcode.com/paper/deep-learning-for-audio-transcription-on-low
Repo
Framework


Title	Story Disambiguation: Tracking Evolving News Stories across News and Social Streams
Authors	Bichen Shi, Thanh-Binh Le, Neil Hurley, Georgiana Ifrim
Abstract	Following a particular news story online is an important but difficult task, as the relevant information is often scattered across different domains/sources (e.g., news articles, blogs, comments, tweets), presented in various formats and language styles, and may overlap with thousands of other stories. In this work we join the areas of topic tracking and entity disambiguation, and propose a framework named Story Disambiguation - a cross-domain story tracking approach that builds on real-time entity disambiguation and a learning-to-rank framework to represent and update the rich semantic structure of news stories. Given a target news story, specified by a seed set of documents, the goal is to effectively select new story-relevant documents from an incoming document stream. We represent stories as entity graphs and we model the story tracking problem as a learning-to-rank task. This enables us to track content with high accuracy, from multiple domains, in real-time. We study a range of text, entity and graph based features to understand which type of features are most effective for representing stories. We further propose new semi-supervised learning techniques to automatically update the story representation over time. Our empirical study shows that we outperform the accuracy of state-of-the-art methods for tracking mixed-domain document streams, while requiring fewer labeled data to seed the tracked stories. This is particularly the case for local news stories that are easily over shadowed by other trending stories, and for complex news stories with ambiguous content in noisy stream environments.
Tasks	Entity Disambiguation, Learning-To-Rank
Published	2018-08-16
URL	http://arxiv.org/abs/1808.05906v1
PDF	http://arxiv.org/pdf/1808.05906v1.pdf
PWC	https://paperswithcode.com/paper/story-disambiguation-tracking-evolving-news
Repo
Framework

Analysis of Langevin Monte Carlo via convex optimization


Title	Analysis of Langevin Monte Carlo via convex optimization
Authors	Alain Durmus, Szymon Majewski, Błażej Miasojedow
Abstract	In this paper, we provide new insights on the Unadjusted Langevin Algorithm. We show that this method can be formulated as a first order optimization algorithm of an objective functional defined on the Wasserstein space of order $2$. Using this interpretation and techniques borrowed from convex optimization, we give a non-asymptotic analysis of this method to sample from logconcave smooth target distribution on $\mathbb{R}^d$. Based on this interpretation, we propose two new methods for sampling from a non-smooth target distribution, which we analyze as well. Besides, these new algorithms are natural extensions of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm, which is a popular extension of the Unadjusted Langevin Algorithm. Similar to SGLD, they only rely on approximations of the gradient of the target log density and can be used for large-scale Bayesian inference.
Tasks	Bayesian Inference
Published	2018-02-26
URL	http://arxiv.org/abs/1802.09188v2
PDF	http://arxiv.org/pdf/1802.09188v2.pdf
PWC	https://paperswithcode.com/paper/analysis-of-langevin-monte-carlo-via-convex
Repo
Framework

Super-Resolution via Conditional Implicit Maximum Likelihood Estimation


Title	Super-Resolution via Conditional Implicit Maximum Likelihood Estimation
Authors	Ke Li, Shichong Peng, Jitendra Malik
Abstract	Single-image super-resolution (SISR) is a canonical problem with diverse applications. Leading methods like SRGAN produce images that contain various artifacts, such as high-frequency noise, hallucinated colours and shape distortions, which adversely affect the realism of the result. In this paper, we propose an alternative approach based on an extension of the method of Implicit Maximum Likelihood Estimation (IMLE). We demonstrate greater effectiveness at noise reduction and preservation of the original colours and shapes, yielding more realistic super-resolved images.
Tasks	Image Super-Resolution, Super-Resolution
Published	2018-10-02
URL	http://arxiv.org/abs/1810.01406v1
PDF	http://arxiv.org/pdf/1810.01406v1.pdf
PWC	https://paperswithcode.com/paper/super-resolution-via-conditional-implicit
Repo
Framework

Counting to Explore and Generalize in Text-based Games


Title	Counting to Explore and Generalize in Text-based Games
Authors	Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
Abstract	We propose a recurrent RL agent with an episodic exploration mechanism that helps discovering good policies in text-based game environments. We show promising results on a set of generated text-based games of varying difficulty where the goal is to collect a coin located at the end of a chain of rooms. In contrast to previous text-based RL approaches, we observe that our agent learns policies that generalize to unseen games of greater difficulty.
Tasks
Published	2018-06-29
URL	http://arxiv.org/abs/1806.11525v2
PDF	http://arxiv.org/pdf/1806.11525v2.pdf
PWC	https://paperswithcode.com/paper/counting-to-explore-and-generalize-in-text
Repo
Framework

Hierarchical clustering with deep Q-learning


Title	Hierarchical clustering with deep Q-learning
Authors	Richard Forster, Agnes Fulop
Abstract	The reconstruction and analyzation of high energy particle physics data is just as important as the analyzation of the structure in real world networks. In a previous study it was explored how hierarchical clustering algorithms can be combined with kt cluster algorithms to provide a more generic clusterization method. Building on that, this paper explores the possibilities to involve deep learning in the process of cluster computation, by applying reinforcement learning techniques. The result is a model, that by learning on a modest dataset of 10; 000 nodes during 70 epochs can reach 83; 77% precision in predicting the appropriate clusters.
Tasks	Q-Learning
Published	2018-05-28
URL	http://arxiv.org/abs/1805.10900v1
PDF	http://arxiv.org/pdf/1805.10900v1.pdf
PWC	https://paperswithcode.com/paper/hierarchical-clustering-with-deep-q-learning
Repo
Framework

Sequenced-Replacement Sampling for Deep Learning


Title	Sequenced-Replacement Sampling for Deep Learning
Authors	Chiu Man Ho, Dae Hoon Park, Wei Yang, Yi Chang
Abstract	We propose sequenced-replacement sampling (SRS) for training deep neural networks. The basic idea is to assign a fixed sequence index to each sample in the dataset. Once a mini-batch is randomly drawn in each training iteration, we refill the original dataset by successively adding samples according to their sequence index. Thus we carry out replacement sampling but in a batched and sequenced way. In a sense, SRS could be viewed as a way of performing “mini-batch augmentation”. It is particularly useful for a task where we have a relatively small images-per-class such as CIFAR-100. Together with a longer period of initial large learning rate, it significantly improves the classification accuracy in CIFAR-100 over the current state-of-the-art results. Our experiments indicate that training deeper networks with SRS is less prone to over-fitting. In the best case, we achieve an error rate as low as 10.10%.
Tasks
Published	2018-10-19
URL	http://arxiv.org/abs/1810.08322v1
PDF	http://arxiv.org/pdf/1810.08322v1.pdf
PWC	https://paperswithcode.com/paper/sequenced-replacement-sampling-for-deep
Repo
Framework

A Tutorial for Weighted Bipolar Argumentation with Continuous Dynamical Systems and the Java Library Attractor


Title	A Tutorial for Weighted Bipolar Argumentation with Continuous Dynamical Systems and the Java Library Attractor
Authors	Nico Potyka
Abstract	Weighted bipolar argumentation frameworks allow modeling decision problems and online discussions by defining arguments and their relationships. The strength of arguments can be computed based on an initial weight and the strength of attacking and supporting arguments. While previous approaches assumed an acyclic argumentation graph and successively set arguments’ strength based on the strength of their parents, recently continuous dynamical systems have been proposed as an alternative. Continuous models update arguments’ strength simultaneously and continuously. While there are currently no analytical guarantees for convergence in general graphs, experiments show that continuous models can converge quickly in large cyclic graphs with thousands of arguments. Here, we focus on the high-level ideas of this approach and explain key results and applications. We also introduce Attractor, a Java library that can be used to solve weighted bipolar argumentation problems. Attractor contains implementations of several discrete and continuous models and numerical algorithms to compute solutions. It also provides base classes that can be used to implement, to evaluate and to compare continuous models easily.
Tasks
Published	2018-11-30
URL	http://arxiv.org/abs/1811.12787v1
PDF	http://arxiv.org/pdf/1811.12787v1.pdf
PWC	https://paperswithcode.com/paper/a-tutorial-for-weighted-bipolar-argumentation
Repo
Framework

Deep multi-survey classification of variable stars


Title	Deep multi-survey classification of variable stars
Authors	Carlos Aguirre, Karim Pichara, Ignacio Becker
Abstract	During the last decade, a considerable amount of effort has been made to classify variable stars using different machine learning techniques. Typically, light curves are represented as vectors of statistical descriptors or features that are used to train various algorithms. These features demand big computational powers that can last from hours to days, making impossible to create scalable and efficient ways of automatically classifying variable stars. Also, light curves from different surveys cannot be integrated and analyzed together when using features, because of observational differences. For example, having variations in cadence and filters, feature distributions become biased and require expensive data-calibration models. The vast amount of data that will be generated soon make necessary to develop scalable machine learning architectures without expensive integration techniques. Convolutional Neural Networks have shown impressing results in raw image classification and representation within the machine learning literature. In this work, we present a novel Deep Learning model for light curve classification, mainly based on convolutional units. Our architecture receives as input the differences between time and magnitude of light curves. It captures the essential classification patterns regardless of cadence and filter. In addition, we introduce a novel data augmentation schema for unevenly sampled time series. We test our method using three different surveys: OGLE-III; Corot; and VVV, which differ in filters, cadence, and area of the sky. We show that besides the benefit of scalability, our model obtains state of the art levels accuracy in light curve classification benchmarks.
Tasks	Calibration, Classification Of Variable Stars, Data Augmentation, Image Classification, Time Series
Published	2018-10-21
URL	http://arxiv.org/abs/1810.09440v1
PDF	http://arxiv.org/pdf/1810.09440v1.pdf
PWC	https://paperswithcode.com/paper/deep-multi-survey-classification-of-variable
Repo
Framework

A Unified Learning Based Framework for Light Field Reconstruction from Coded Projections


Title	A Unified Learning Based Framework for Light Field Reconstruction from Coded Projections
Authors	Anil Kumar Vadathya, Sharath Girish, Kaushik Mitra
Abstract	Light field presents a rich way to represent the 3D world by capturing the spatio-angular dimensions of the visual signal. However, the popular way of capturing light field (LF) via a plenoptic camera presents spatio-angular resolution trade-off. Computational imaging techniques such as compressive light field and programmable coded aperture reconstruct full sensor resolution LF from coded projections obtained by multiplexing the incoming spatio-angular light field. Here, we present a unified learning framework that can reconstruct LF from a variety of multiplexing schemes with minimal number of coded images as input. We consider three light field capture schemes: heterodyne capture scheme with code placed near the sensor, coded aperture scheme with code at the camera aperture and finally the dual exposure scheme of capturing a focus-defocus pair where there is no explicit coding. Our algorithm consists of three stages 1) we recover the all-in-focus image from the coded image 2) we estimate the disparity maps for all the LF views from the coded image and the all-in-focus image, 3) we then render the LF by warping the all-in-focus image using disparity maps and refine it. For these three stages we propose three deep neural networks - ViewNet, DispairtyNet and RefineNet. Our reconstructions show that our learning algorithm achieves state-of-the-art results for all the three multiplexing schemes. Especially, our LF reconstructions from focus-defocus pair is comparable to other learning-based view synthesis approaches from multiple images. Thus, our work paves the way for capturing high-resolution LF (~ a megapixel) using conventional cameras such as DSLRs. Please check our supplementary materials $\href{https://docs.google.com/presentation/d/1Vr-F8ZskrSd63tvnLfJ2xmEXY6OBc1Rll3XeOAtc11I/}{online}$ to better appreciate the reconstructed light fields.
Tasks
Published	2018-12-26
URL	https://arxiv.org/abs/1812.10532v2
PDF	https://arxiv.org/pdf/1812.10532v2.pdf
PWC	https://paperswithcode.com/paper/a-unified-learning-based-framework-for-light
Repo
Framework

Deep Reinforcement Learning of Marked Temporal Point Processes


Title	Deep Reinforcement Learning of Marked Temporal Point Processes
Authors	Utkarsh Upadhyay, Abir De, Manuel Gomez-Rodriguez
Abstract	In a wide variety of applications, humans interact with a complex environment by means of asynchronous stochastic discrete events in continuous time. Can we design online interventions that will help humans achieve certain goals in such asynchronous setting? In this paper, we address the above problem from the perspective of deep reinforcement learning of marked temporal point processes, where both the actions taken by an agent and the feedback it receives from the environment are asynchronous stochastic discrete events characterized using marked temporal point processes. In doing so, we define the agent’s policy using the intensity and mark distribution of the corresponding process and then derive a flexible policy gradient method, which embeds the agent’s actions and the feedback it receives into real-valued vectors using deep recurrent neural networks. Our method does not make any assumptions on the functional form of the intensity and mark distribution of the feedback and it allows for arbitrarily complex reward functions. We apply our methodology to two different applications in personalized teaching and viral marketing and, using data gathered from Duolingo and Twitter, we show that it may be able to find interventions to help learners and marketers achieve their goals more effectively than alternatives.
Tasks	Point Processes
Published	2018-05-23
URL	http://arxiv.org/abs/1805.09360v2
PDF	http://arxiv.org/pdf/1805.09360v2.pdf
PWC	https://paperswithcode.com/paper/deep-reinforcement-learning-of-marked
Repo
Framework

Swarming for Faster Convergence in Stochastic Optimization


Title	Swarming for Faster Convergence in Stochastic Optimization
Authors	Shi Pu, Alfredo Garcia
Abstract	We study a distributed framework for stochastic optimization which is inspired by models of collective motion found in nature (e.g., swarming) with mild communication requirements. Specifically, we analyze a scheme in which each one of $N > 1$ independent threads, implements in a distributed and unsynchronized fashion, a stochastic gradient-descent algorithm which is perturbed by a swarming potential. Assuming the overhead caused by synchronization is not negligible, we show the swarming-based approach exhibits better performance than a centralized algorithm (based upon the average of $N$ observations) in terms of (real-time) convergence speed. We also derive an error bound that is monotone decreasing in network size and connectivity. We characterize the scheme’s finite-time performances for both convex and non-convex objective functions.
Tasks	Stochastic Optimization
Published	2018-06-11
URL	http://arxiv.org/abs/1806.04207v2
PDF	http://arxiv.org/pdf/1806.04207v2.pdf
PWC	https://paperswithcode.com/paper/swarming-for-faster-convergence-in-stochastic
Repo
Framework

Tensor Decomposition for Compressing Recurrent Neural Network


Title	Tensor Decomposition for Compressing Recurrent Neural Network
Authors	Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Abstract	In the machine learning fields, Recurrent Neural Network (RNN) has become a popular architecture for sequential data modeling. However, behind the impressive performance, RNNs require a large number of parameters for both training and inference. In this paper, we are trying to reduce the number of parameters and maintain the expressive power from RNN simultaneously. We utilize several tensor decompositions method including CANDECOMP/PARAFAC (CP), Tucker decomposition and Tensor Train (TT) to re-parameterize the Gated Recurrent Unit (GRU) RNN. We evaluate all tensor-based RNNs performance on sequence modeling tasks with a various number of parameters. Based on our experiment results, TT-GRU achieved the best results in a various number of parameters compared to other decomposition methods.
Tasks
Published	2018-02-28
URL	http://arxiv.org/abs/1802.10410v2
PDF	http://arxiv.org/pdf/1802.10410v2.pdf
PWC	https://paperswithcode.com/paper/tensor-decomposition-for-compressing
Repo
Framework

Nonlinear Inductive Matrix Completion based on One-layer Neural Networks


Title	Nonlinear Inductive Matrix Completion based on One-layer Neural Networks
Authors	Kai Zhong, Zhao Song, Prateek Jain, Inderjit S. Dhillon
Abstract	The goal of a recommendation system is to predict the interest of a user in a given item by exploiting the existing set of ratings as well as certain user/item features. A standard approach to modeling this problem is Inductive Matrix Completion where the predicted rating is modeled as an inner product of the user and the item features projected onto a latent space. In order to learn the parameters effectively from a small number of observed ratings, the latent space is constrained to be low-dimensional which implies that the parameter matrix is constrained to be low-rank. However, such bilinear modeling of the ratings can be limiting in practice and non-linear prediction functions can lead to significant improvements. A natural approach to introducing non-linearity in the prediction function is to apply a non-linear activation function on top of the projected user/item features. Imposition of non-linearities further complicates an already challenging problem that has two sources of non-convexity: a) low-rank structure of the parameter matrix, and b) non-linear activation function. We show that one can still solve the non-linear Inductive Matrix Completion problem using gradient descent type methods as long as the solution is initialized well. That is, close to the optima, the optimization function is strongly convex and hence admits standard optimization techniques, at least for certain activation functions, such as Sigmoid and tanh. We also highlight the importance of the activation function and show how ReLU can behave significantly differently than say a sigmoid function. Finally, we apply our proposed technique to recommendation systems and semi-supervised clustering, and show that our method can lead to much better performance than standard linear Inductive Matrix Completion methods.
Tasks	Matrix Completion, Recommendation Systems
Published	2018-05-26
URL	http://arxiv.org/abs/1805.10477v1
PDF	http://arxiv.org/pdf/1805.10477v1.pdf
PWC	https://paperswithcode.com/paper/nonlinear-inductive-matrix-completion-based
Repo
Framework

Tracking Emerges by Colorizing Videos


Title	Tracking Emerges by Colorizing Videos
Authors	Carl Vondrick, Abhinav Shrivastava, Alireza Fathi, Sergio Guadarrama, Kevin Murphy
Abstract	We use large amounts of unlabeled video to learn models for visual tracking without manual human supervision. We leverage the natural temporal coherency of color to create a model that learns to colorize gray-scale videos by copying colors from a reference frame. Quantitative and qualitative experiments suggest that this task causes the model to automatically learn to track visual regions. Although the model is trained without any ground-truth labels, our method learns to track well enough to outperform the latest methods based on optical flow. Moreover, our results suggest that failures to track are correlated with failures to colorize, indicating that advancing video colorization may further improve self-supervised visual tracking.
Tasks	Colorization, Optical Flow Estimation, Skeleton Based Action Recognition, Visual Tracking
Published	2018-06-25
URL	http://arxiv.org/abs/1806.09594v2
PDF	http://arxiv.org/pdf/1806.09594v2.pdf
PWC	https://paperswithcode.com/paper/tracking-emerges-by-colorizing-videos
Repo
Framework