July 27, 2019

3061 words 15 mins read

Paper Group ANR 479

Gradient Boosting on Stochastic Data Streams. A Variance Maximization Criterion for Active Learning. Providing Effective Real-time Feedback in Simulation-based Surgical Training. Adaptive Ensemble Prediction for Deep Neural Networks based on Confidence Level. Expert Level control of Ramp Metering based on Multi-task Deep Reinforcement Learning. The …

Gradient Boosting on Stochastic Data Streams


Title	Gradient Boosting on Stochastic Data Streams
Authors	Hanzhang Hu, Wen Sun, Arun Venkatraman, Martial Hebert, J. Andrew Bagnell
Abstract	Boosting is a popular ensemble algorithm that generates more powerful learners by linearly combining base models from a simpler hypothesis class. In this work, we investigate the problem of adapting batch gradient boosting for minimizing convex loss functions to online setting where the loss at each iteration is i.i.d sampled from an unknown distribution. To generalize from batch to online, we first introduce the definition of online weak learning edge with which for strongly convex and smooth loss functions, we present an algorithm, Streaming Gradient Boosting (SGB) with exponential shrinkage guarantees in the number of weak learners. We further present an adaptation of SGB to optimize non-smooth loss functions, for which we derive a O(ln N/N) convergence rate. We also show that our analysis can extend to adversarial online learning setting under a stronger assumption that the online weak learning edge will hold in adversarial setting. We finally demonstrate experimental results showing that in practice our algorithms can achieve competitive results as classic gradient boosting while using less computation.
Tasks
Published	2017-03-01
URL	http://arxiv.org/abs/1703.00377v1
PDF	http://arxiv.org/pdf/1703.00377v1.pdf
PWC	https://paperswithcode.com/paper/gradient-boosting-on-stochastic-data-streams
Repo
Framework

A Variance Maximization Criterion for Active Learning


Title	A Variance Maximization Criterion for Active Learning
Authors	Yazhou Yang, Marco Loog
Abstract	Active learning aims to train a classifier as fast as possible with as few labels as possible. The core element in virtually any active learning strategy is the criterion that measures the usefulness of the unlabeled data based on which new points to be labeled are picked. We propose a novel approach which we refer to as maximizing variance for active learning or MVAL for short. MVAL measures the value of unlabeled instances by evaluating the rate of change of output variables caused by changes in the next sample to be queried and its potential labelling. In a sense, this criterion measures how unstable the classifier’s output is for the unlabeled data points under perturbations of the training data. MVAL maintains, what we refer to as, retraining information matrices to keep track of these output scores and exploits two kinds of variance to measure the informativeness and representativeness, respectively. By fusing these variances, MVAL is able to select the instances which are both informative and representative. We employ our technique both in combination with logistic regression and support vector machines and demonstrate that MVAL achieves state-of-the-art performance in experiments on a large number of standard benchmark datasets.
Tasks	Active Learning
Published	2017-06-23
URL	http://arxiv.org/abs/1706.07642v2
PDF	http://arxiv.org/pdf/1706.07642v2.pdf
PWC	https://paperswithcode.com/paper/a-variance-maximization-criterion-for-active
Repo
Framework

Providing Effective Real-time Feedback in Simulation-based Surgical Training


Title	Providing Effective Real-time Feedback in Simulation-based Surgical Training
Authors	Xingjun Ma, Sudanthi Wijewickrema, Yun Zhou, Shuo Zhou, Stephen O’Leary, James Bailey
Abstract	Virtual reality simulation is becoming popular as a training platform in surgical education. However, one important aspect of simulation-based surgical training that has not received much attention is the provision of automated real-time performance feedback to support the learning process. Performance feedback is actionable advice that improves novice behaviour. In simulation, automated feedback is typically extracted from prediction models trained using data mining techniques. Existing techniques suffer from either low effectiveness or low efficiency resulting in their inability to be used in real-time. In this paper, we propose a random forest based method that finds a balance between effectiveness and efficiency. Experimental results in a temporal bone surgery simulation show that the proposed method is able to extract highly effective feedback at a high level of efficiency.
Tasks
Published	2017-06-30
URL	http://arxiv.org/abs/1706.10036v1
PDF	http://arxiv.org/pdf/1706.10036v1.pdf
PWC	https://paperswithcode.com/paper/providing-effective-real-time-feedback-in
Repo
Framework

Adaptive Ensemble Prediction for Deep Neural Networks based on Confidence Level


Title	Adaptive Ensemble Prediction for Deep Neural Networks based on Confidence Level
Authors	Hiroshi Inoue
Abstract	Ensembling multiple predictions is a widely used technique for improving the accuracy of various machine learning tasks. One obvious drawback of ensembling is its higher execution cost during inference. In this paper, we first describe our insights on the relationship between the probability of prediction and the effect of ensembling with current deep neural networks; ensembling does not help mispredictions for inputs predicted with a high probability even when there is a non-negligible number of mispredicted inputs. This finding motivated us to develop a way to adaptively control the ensembling. If the prediction for an input reaches a high enough probability, i.e., the output from the softmax function, on the basis of the confidence level, we stop ensembling for this input to avoid wasting computation power. We evaluated the adaptive ensembling by using various datasets and showed that it reduces the computation cost significantly while achieving accuracy similar to that of static ensembling using a pre-defined number of local predictions. We also show that our statistically rigorous confidence-level-based early-exit condition reduces the burden of task-dependent threshold tuning better compared with naive early exit based on a pre-defined threshold in addition to yielding a better accuracy with the same cost.
Tasks
Published	2017-02-27
URL	http://arxiv.org/abs/1702.08259v3
PDF	http://arxiv.org/pdf/1702.08259v3.pdf
PWC	https://paperswithcode.com/paper/fast-and-accurate-inference-with-adaptive
Repo
Framework

Expert Level control of Ramp Metering based on Multi-task Deep Reinforcement Learning


Title	Expert Level control of Ramp Metering based on Multi-task Deep Reinforcement Learning
Authors	Francois Belletti, Daniel Haziza, Gabriel Gomes, Alexandre M. Bayen
Abstract	This article shows how the recent breakthroughs in Reinforcement Learning (RL) that have enabled robots to learn to play arcade video games, walk or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear Partial Differential Equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. We show how neural network based RL enables the control of discretized PDEs whose parameters are unknown, random, and time-varying. We introduce an algorithm of Mutual Weight Regularization (MWR) which alleviates the curse of dimensionality of multi-agent control schemes by sharing experience between agents while giving each agent the opportunity to specialize its action policy so as to tailor it to the local parameters of the part of the system it is located in.
Tasks
Published	2017-01-30
URL	http://arxiv.org/abs/1701.08832v1
PDF	http://arxiv.org/pdf/1701.08832v1.pdf
PWC	https://paperswithcode.com/paper/expert-level-control-of-ramp-metering-based
Repo
Framework

The Effect of Color Space Selection on Detectability and Discriminability of Colored Objects


Title	The Effect of Color Space Selection on Detectability and Discriminability of Colored Objects
Authors	Amir Rasouli, John K. Tsotsos
Abstract	In this paper, we investigate the effect of color space selection on detectability and discriminability of colored objects under various conditions. 20 color spaces from the literature are evaluated on a large dataset of simulated and real images. We measure the suitability of color spaces from two different perspectives: detectability and discriminability of various color groups. Through experimental evaluation, we found that there is no single optimal color space suitable for all color groups. The color spaces have different levels of sensitivity to different color groups and they are useful depending on the color of the sought object. Overall, the best results were achieved in both simulated and real images using color spaces C1C2C3, UVW and XYZ. In addition, using a simulated environment, we show a practical application of color space selection in the context of top-down control in active visual search. The results indicate that on average color space C1C2C3 followed by HSI and XYZ achieve the best time in searching for objects of various colors. Here, the right choice of color space can improve time of search on average by 20%. As part of our contribution, we also introduce a large dataset of simulated 3D objects
Tasks
Published	2017-02-14
URL	http://arxiv.org/abs/1702.05421v1
PDF	http://arxiv.org/pdf/1702.05421v1.pdf
PWC	https://paperswithcode.com/paper/the-effect-of-color-space-selection-on
Repo
Framework

Recurrent Ladder Networks


Title	Recurrent Ladder Networks
Authors	Isabeau Prémont-Schwarz, Alexander Ilin, Tele Hotloo Hao, Antti Rasmus, Rinu Boney, Harri Valpola
Abstract	We propose a recurrent extension of the Ladder networks whose structure is motivated by the inference required in hierarchical latent variable models. We demonstrate that the recurrent Ladder is able to handle a wide variety of complex learning tasks that benefit from iterative inference and temporal modeling. The architecture shows close-to-optimal results on temporal modeling of video data, competitive results on music modeling, and improved perceptual grouping based on higher order abstractions, such as stochastic textures and motion cues. We present results for fully supervised, semi-supervised, and unsupervised tasks. The results suggest that the proposed architecture and principles are powerful tools for learning a hierarchy of abstractions, learning iterative inference and handling temporal information.
Tasks	Latent Variable Models, Music Modeling
Published	2017-07-28
URL	http://arxiv.org/abs/1707.09219v4
PDF	http://arxiv.org/pdf/1707.09219v4.pdf
PWC	https://paperswithcode.com/paper/recurrent-ladder-networks
Repo
Framework

Games and Big Data: A Scalable Multi-Dimensional Churn Prediction Model


Title	Games and Big Data: A Scalable Multi-Dimensional Churn Prediction Model
Authors	Paul Bertens, Anna Guitart, África Periáñez
Abstract	The emergence of mobile games has caused a paradigm shift in the video-game industry. Game developers now have at their disposal a plethora of information on their players, and thus can take advantage of reliable models that can accurately predict player behavior and scale to huge datasets. Churn prediction, a challenge common to a variety of sectors, is particularly relevant for the mobile game industry, as player retention is crucial for the successful monetization of a game. In this article, we present an approach to predicting game abandon based on survival ensembles. Our method provides accurate predictions on both the level at which each player will leave the game and their accumulated playtime until that moment. Further, it is robust to different data distributions and applicable to a wide range of response variables, while also allowing for efficient parallelization of the algorithm. This makes our model well suited to perform real-time analyses of churners, even for games with millions of daily active users.
Tasks
Published	2017-10-06
URL	http://arxiv.org/abs/1710.02262v1
PDF	http://arxiv.org/pdf/1710.02262v1.pdf
PWC	https://paperswithcode.com/paper/games-and-big-data-a-scalable-multi
Repo
Framework

Weakly Supervised Dense Video Captioning


Title	Weakly Supervised Dense Video Captioning
Authors	Zhiqiang Shen, Jianguo Li, Zhou Su, Minjun Li, Yurong Chen, Yu-Gang Jiang, Xiangyang Xue
Abstract	This paper focuses on a novel and challenging vision task, dense video captioning, which aims to automatically describe a video clip with multiple informative and diverse caption sentences. The proposed method is trained without explicit annotation of fine-grained sentence to video region-sequence correspondence, but is only based on weak video-level sentence annotations. It differs from existing video captioning systems in three technical aspects. First, we propose lexical fully convolutional neural networks (Lexical-FCN) with weakly supervised multi-instance multi-label learning to weakly link video regions with lexical labels. Second, we introduce a novel submodular maximization scheme to generate multiple informative and diverse region-sequences based on the Lexical-FCN outputs. A winner-takes-all scheme is adopted to weakly associate sentences to region-sequences in the training phase. Third, a sequence-to-sequence learning based language model is trained with the weakly supervised information obtained through the association process. We show that the proposed method can not only produce informative and diverse dense captions, but also outperform state-of-the-art single video captioning methods by a large margin.
Tasks	Dense Video Captioning, Language Modelling, Multi-Label Learning, Video Captioning
Published	2017-04-05
URL	http://arxiv.org/abs/1704.01502v1
PDF	http://arxiv.org/pdf/1704.01502v1.pdf
PWC	https://paperswithcode.com/paper/weakly-supervised-dense-video-captioning
Repo
Framework

ChaLearn Looking at People: A Review of Events and Resources


Title	ChaLearn Looking at People: A Review of Events and Resources
Authors	Sergio Escalera, Xavier Baró, Hugo Jair Escalante, Isabelle Guyon
Abstract	This paper reviews the historic of ChaLearn Looking at People (LAP) events. We started in 2011 (with the release of the first Kinect device) to run challenges related to human action/activity and gesture recognition. Since then we have regularly organized events in a series of competitions covering all aspects of visual analysis of humans. So far we have organized more than 10 international challenges and events in this field. This paper reviews associated events, and introduces the ChaLearn LAP platform where public resources (including code, data and preprints of papers) related to the organized events are available. We also provide a discussion on perspectives of ChaLearn LAP activities.
Tasks	Gesture Recognition
Published	2017-01-10
URL	http://arxiv.org/abs/1701.02664v2
PDF	http://arxiv.org/pdf/1701.02664v2.pdf
PWC	https://paperswithcode.com/paper/chalearn-looking-at-people-a-review-of-events
Repo
Framework

Computer-aided position planning of miniplates to treat facial bone defects


Title	Computer-aided position planning of miniplates to treat facial bone defects
Authors	Jan Egger, Jürgen Wallner, Markus Gall, Xiaojun Chen, Katja Schwenzer-Zimmerer, Knut Reinbacher, Dieter Schmalstieg
Abstract	In this contribution, a software system for computer-aided position planning of miniplates to treat facial bone defects is proposed. The intra-operatively used bone plates have to be passively adapted on the underlying bone contours for adequate bone fragment stabilization. However, this procedure can lead to frequent intra-operatively performed material readjustments especially in complex surgical cases. Our approach is able to fit a selection of common implant models on the surgeon’s desired position in a 3D computer model. This happens with respect to the surrounding anatomical structures, always including the possibility of adjusting both the direction and the position of the used osteosynthesis material. By using the proposed software, surgeons are able to pre-plan the out coming implant in its form and morphology with the aid of a computer-visualized model within a few minutes. Further, the resulting model can be stored in STL file format, the commonly used format for 3D printing. Using this technology, surgeons are able to print the virtual generated implant, or create an individually designed bending tool. This method leads to adapted osteosynthesis materials according to the surrounding anatomy and requires further a minimum amount of money and time.
Tasks
Published	2017-08-17
URL	http://arxiv.org/abs/1708.05711v1
PDF	http://arxiv.org/pdf/1708.05711v1.pdf
PWC	https://paperswithcode.com/paper/computer-aided-position-planning-of
Repo
Framework

Tuning Free Orthogonal Matching Pursuit


Title	Tuning Free Orthogonal Matching Pursuit
Authors	Sreejith Kallummil, Sheetal Kalyani
Abstract	Orthogonal matching pursuit (OMP) is a widely used compressive sensing (CS) algorithm for recovering sparse signals in noisy linear regression models. The performance of OMP depends on its stopping criteria (SC). SC for OMP discussed in literature typically assumes knowledge of either the sparsity of the signal to be estimated $k_0$ or noise variance $\sigma^2$, both of which are unavailable in many practical applications. In this article we develop a modified version of OMP called tuning free OMP or TF-OMP which does not require a SC. TF-OMP is proved to accomplish successful sparse recovery under the usual assumptions on restricted isometry constants (RIC) and mutual coherence of design matrix. TF-OMP is numerically shown to deliver a highly competitive performance in comparison with OMP having \textit{a priori} knowledge of $k_0$ or $\sigma^2$. Greedy algorithm for robust de-noising (GARD) is an OMP like algorithm proposed for efficient estimation in classical overdetermined linear regression models corrupted by sparse outliers. However, GARD requires the knowledge of inlier noise variance which is difficult to estimate. We also produce a tuning free algorithm (TF-GARD) for efficient estimation in the presence of sparse outliers by extending the operating principle of TF-OMP to GARD. TF-GARD is numerically shown to achieve a performance comparable to that of the existing implementation of GARD.
Tasks	Compressive Sensing
Published	2017-03-15
URL	http://arxiv.org/abs/1703.05080v1
PDF	http://arxiv.org/pdf/1703.05080v1.pdf
PWC	https://paperswithcode.com/paper/tuning-free-orthogonal-matching-pursuit
Repo
Framework

Automatic Speech Recognition with Very Large Conversational Finnish and Estonian Vocabularies


Title	Automatic Speech Recognition with Very Large Conversational Finnish and Estonian Vocabularies
Authors	Seppo Enarvi, Peter Smit, Sami Virpioja, Mikko Kurimo
Abstract	Today, the vocabulary size for language models in large vocabulary speech recognition is typically several hundreds of thousands of words. While this is already sufficient in some applications, the out-of-vocabulary words are still limiting the usability in others. In agglutinative languages the vocabulary for conversational speech should include millions of word forms to cover the spelling variations due to colloquial pronunciations, in addition to the word compounding and inflections. Very large vocabularies are also needed, for example, when the recognition of rare proper names is important.
Tasks	Large Vocabulary Continuous Speech Recognition, Speech Recognition
Published	2017-07-13
URL	http://arxiv.org/abs/1707.04227v5
PDF	http://arxiv.org/pdf/1707.04227v5.pdf
PWC	https://paperswithcode.com/paper/automatic-speech-recognition-with-very-large
Repo
Framework

Accelerated Optimization in the PDE Framework: Formulations for the Active Contour Case


Title	Accelerated Optimization in the PDE Framework: Formulations for the Active Contour Case
Authors	Anthony Yezzi, Ganesh Sundaramoorthi
Abstract	Following the seminal work of Nesterov, accelerated optimization methods have been used to powerfully boost the performance of first-order, gradient-based parameter estimation in scenarios where second-order optimization strategies are either inapplicable or impractical. Not only does accelerated gradient descent converge considerably faster than traditional gradient descent, but it also performs a more robust local search of the parameter space by initially overshooting and then oscillating back as it settles into a final configuration, thereby selecting only local minimizers with a basis of attraction large enough to contain the initial overshoot. This behavior has made accelerated and stochastic gradient search methods particularly popular within the machine learning community. In their recent PNAS 2016 paper, Wibisono, Wilson, and Jordan demonstrate how a broad class of accelerated schemes can be cast in a variational framework formulated around the Bregman divergence, leading to continuum limit ODE’s. We show how their formulation may be further extended to infinite dimension manifolds (starting here with the geometric space of curves and surfaces) by substituting the Bregman divergence with inner products on the tangent space and explicitly introducing a distributed mass model which evolves in conjunction with the object of interest during the optimization process. The co-evolving mass model, which is introduced purely for the sake of endowing the optimization with helpful dynamics, also links the resulting class of accelerated PDE based optimization schemes to fluid dynamical formulations of optimal mass transport.
Tasks
Published	2017-11-27
URL	http://arxiv.org/abs/1711.09867v1
PDF	http://arxiv.org/pdf/1711.09867v1.pdf
PWC	https://paperswithcode.com/paper/accelerated-optimization-in-the-pde-framework-1
Repo
Framework

Machine learning in sentiment reconstruction of the simulated stock market


Title	Machine learning in sentiment reconstruction of the simulated stock market
Authors	Mikhail Goykhman, Ali Teimouri
Abstract	In this paper we continue the study of the simulated stock market framework defined by the driving sentiment processes. We focus on the market environment driven by the buy/sell trading sentiment process of the Markov chain type. We apply the methodology of the Hidden Markov Models and the Recurrent Neural Networks to reconstruct the transition probabilities matrix of the Markov sentiment process and recover the underlying sentiment states from the observed stock price behavior.
Tasks
Published	2017-08-06
URL	http://arxiv.org/abs/1708.01897v1
PDF	http://arxiv.org/pdf/1708.01897v1.pdf
PWC	https://paperswithcode.com/paper/machine-learning-in-sentiment-reconstruction
Repo
Framework