October 17, 2019

2922 words 14 mins read

Paper Group ANR 720

Algorithms for solving optimization problems arising from deep neural net models: smooth problems. NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning. Inflection-Tolerant Ontology-Based Named Entity Recognition for Real-Time Applications. Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment. A Multi-Sta …

Algorithms for solving optimization problems arising from deep neural net models: smooth problems


Title	Algorithms for solving optimization problems arising from deep neural net models: smooth problems
Authors	Vyacheslav Kungurtsev, Tomas Pevny
Abstract	Machine Learning models incorporating multiple layered learning networks have been seen to provide effective models for various classification problems. The resulting optimization problem to solve for the optimal vector minimizing the empirical risk is, however, highly nonlinear. This presents a challenge to application and development of appropriate optimization algorithms for solving the problem. In this paper, we summarize the primary challenges involved and present the case for a Newton-based method incorporating directions of negative curvature, including promising numerical results on data arising from security anomally deetection.
Tasks
Published	2018-06-30
URL	http://arxiv.org/abs/1807.00172v1
PDF	http://arxiv.org/pdf/1807.00172v1.pdf
PWC	https://paperswithcode.com/paper/algorithms-for-solving-optimization-problems-1
Repo
Framework

NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning


Title	NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning
Authors	Alexander Richard, Hilde Kuehne, Ahsan Iqbal, Juergen Gall
Abstract	Video learning is an important task in computer vision and has experienced increasing interest over the recent years. Since even a small amount of videos easily comprises several million frames, methods that do not rely on a frame-level annotation are of special importance. In this work, we propose a novel learning algorithm with a Viterbi-based loss that allows for online and incremental learning of weakly annotated video data. We moreover show that explicit context and length modeling leads to huge improvements in video segmentation and labeling tasks andinclude these models into our framework. On several action segmentation benchmarks, we obtain an improvement of up to 10% compared to current state-of-the-art methods.
Tasks	action segmentation, Video Semantic Segmentation
Published	2018-05-17
URL	http://arxiv.org/abs/1805.06875v1
PDF	http://arxiv.org/pdf/1805.06875v1.pdf
PWC	https://paperswithcode.com/paper/neuralnetwork-viterbi-a-framework-for-weakly
Repo
Framework

Inflection-Tolerant Ontology-Based Named Entity Recognition for Real-Time Applications


Title	Inflection-Tolerant Ontology-Based Named Entity Recognition for Real-Time Applications
Authors	Christian Jilek, Markus Schröder, Rudolf Novik, Sven Schwarz, Heiko Maus, Andreas Dengel
Abstract	A growing number of applications users daily interact with have to operate in (near) real-time: chatbots, digital companions, knowledge work support systems – just to name a few. To perform the services desired by the user, these systems have to analyze user activity logs or explicit user input extremely fast. In particular, text content (e.g. in form of text snippets) needs to be processed in an information extraction task. Regarding the aforementioned temporal requirements, this has to be accomplished in just a few milliseconds, which limits the number of methods that can be applied. Practically, only very fast methods remain, which on the other hand deliver worse results than slower but more sophisticated Natural Language Processing (NLP) pipelines. In this paper, we investigate and propose methods for real-time capable Named Entity Recognition (NER). As a first improvement step we address are word variations induced by inflection, for example present in the German language. Our approach is ontology-based and makes use of several language information sources like Wiktionary. We evaluated it using the German Wikipedia (about 9.4B characters), for which the whole NER process took considerably less than an hour. Since precision and recall are higher than with comparably fast methods, we conclude that the quality gap between high speed methods and sophisticated NLP pipelines can be narrowed a bit more without losing too much runtime performance.
Tasks	Named Entity Recognition
Published	2018-12-05
URL	http://arxiv.org/abs/1812.02119v1
PDF	http://arxiv.org/pdf/1812.02119v1.pdf
PWC	https://paperswithcode.com/paper/inflection-tolerant-ontology-based-named
Repo
Framework

Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment


Title	Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
Authors	Li Ding, Chenliang Xu
Abstract	In this work, we address the task of weakly-supervised human action segmentation in long, untrimmed videos. Recent methods have relied on expensive learning models, such as Recurrent Neural Networks (RNN) and Hidden Markov Models (HMM). However, these methods suffer from expensive computational cost, thus are unable to be deployed in large scale. To overcome the limitations, the keys to our design are efficiency and scalability. We propose a novel action modeling framework, which consists of a new temporal convolutional network, named Temporal Convolutional Feature Pyramid Network (TCFPN), for predicting frame-wise action labels, and a novel training strategy for weakly-supervised sequence modeling, named Iterative Soft Boundary Assignment (ISBA), to align action sequences and update the network in an iterative fashion. The proposed framework is evaluated on two benchmark datasets, Breakfast and Hollywood Extended, with four different evaluation metrics. Extensive experimental results show that our methods achieve competitive or superior performance to state-of-the-art methods.
Tasks	action segmentation
Published	2018-03-28
URL	http://arxiv.org/abs/1803.10699v1
PDF	http://arxiv.org/pdf/1803.10699v1.pdf
PWC	https://paperswithcode.com/paper/weakly-supervised-action-segmentation-with
Repo
Framework

A Multi-State Diagnosis and Prognosis Framework with Feature Learning for Tool Condition Monitoring


Title	A Multi-State Diagnosis and Prognosis Framework with Feature Learning for Tool Condition Monitoring
Authors	Chong Zhang, Geok Soon Hong, Jun-Hong Zhou, Kay Chen Tan, Haizhou Li, Huan Xu, Jihoon Hong, Hian-Leng Chan
Abstract	In this paper, a multi-state diagnosis and prognosis (MDP) framework is proposed for tool condition monitoring via a deep belief network based multi-state approach (DBNMS). For fault diagnosis, a cost-sensitive deep belief network (namely ECS-DBN) is applied to deal with the imbalanced data problem for tool state estimation. An appropriate prognostic degradation model is then applied for tool wear estimation based on the different tool states. The proposed framework has the advantage of automatic feature representation learning and shows better performance in accuracy and robustness. The effectiveness of the proposed DBNMS is validated using a real-world dataset obtained from the gun drilling process. This dataset contains a large amount of measured signals involving different tool geometries under various operating conditions. The DBNMS is examined for both the tool state estimation and tool wear estimation tasks. In the experimental studies, the prediction results are evaluated and compared with popular machine learning approaches, which show the superior performance of the proposed DBNMS approach.
Tasks	Representation Learning
Published	2018-04-30
URL	http://arxiv.org/abs/1805.00367v1
PDF	http://arxiv.org/pdf/1805.00367v1.pdf
PWC	https://paperswithcode.com/paper/a-multi-state-diagnosis-and-prognosis
Repo
Framework

Predicting Electricity Outages Caused by Convective Storms


Title	Predicting Electricity Outages Caused by Convective Storms
Authors	Roope Tervo, Joonas Karjalainen, Alexander Jung
Abstract	We consider the problem of predicting power outages in an electrical power grid due to hazards produced by convective storms. These storms produce extreme weather phenomena such as intense wind, tornadoes and lightning over a small area. In this paper, we discuss the application of state-of-the-art machine learning techniques, such as random forest classifiers and deep neural networks, to predict the amount of damage caused by storms. We cast this application as a classification problem where the goal is to classify storm cells into a finite number of classes, each corresponding to a certain amount of expected damage. The classification method use as input features estimates for storm cell location and movement which has to be extracted from the raw data. A main challenge of this application is that the training data is heavily imbalanced as the occurrence of extreme weather events is rare. In order to address this issue, we applied SMOTE technique.
Tasks
Published	2018-05-21
URL	http://arxiv.org/abs/1805.07897v1
PDF	http://arxiv.org/pdf/1805.07897v1.pdf
PWC	https://paperswithcode.com/paper/predicting-electricity-outages-caused-by
Repo
Framework

Fusion of complex networks and randomized neural networks for texture analysis


Title	Fusion of complex networks and randomized neural networks for texture analysis
Authors	Lucas C. Ribas, Jarbas J. M. Sa Junior, Leonardo F. S. Scabini, Odemir M. Bruno
Abstract	This paper presents a high discriminative texture analysis method based on the fusion of complex networks and randomized neural networks. In this approach, the input image is modeled as a complex networks and its topological properties as well as the image pixels are used to train randomized neural networks in order to create a signature that represents the deep characteristics of the texture. The results obtained surpassed the accuracies of many methods available in the literature. This performance demonstrates that our proposed approach opens a promising source of research, which consists of exploring the synergy of neural networks and complex networks in the texture analysis field.
Tasks	Texture Classification
Published	2018-06-24
URL	http://arxiv.org/abs/1806.09170v1
PDF	http://arxiv.org/pdf/1806.09170v1.pdf
PWC	https://paperswithcode.com/paper/fusion-of-complex-networks-and-randomized
Repo
Framework

Subitizing with Variational Autoencoders


Title	Subitizing with Variational Autoencoders
Authors	Rijnder Wever, Tom F. H. Runia
Abstract	Numerosity, the number of objects in a set, is a basic property of a given visual scene. Many animals develop the perceptual ability to subitize: the near-instantaneous identification of the numerosity in small sets of visual items. In computer vision, it has been shown that numerosity emerges as a statistical property in neural networks during unsupervised learning from simple synthetic images. In this work, we focus on more complex natural images using unsupervised hierarchical neural networks. Specifically, we show that variational autoencoders are able to spontaneously perform subitizing after training without supervision on a large amount images from the Salient Object Subitizing dataset. While our method is unable to outperform supervised convolutional networks for subitizing, we observe that the networks learn to encode numerosity as basic visual property. Moreover, we find that the learned representations are likely invariant to object area; an observation in alignment with studies on biological neural networks in cognitive neuroscience.
Tasks
Published	2018-08-01
URL	http://arxiv.org/abs/1808.00257v1
PDF	http://arxiv.org/pdf/1808.00257v1.pdf
PWC	https://paperswithcode.com/paper/subitizing-with-variational-autoencoders
Repo
Framework

Selective Experience Replay for Lifelong Learning


Title	Selective Experience Replay for Lifelong Learning
Authors	David Isele, Akansel Cosgun
Abstract	Deep reinforcement learning has emerged as a powerful tool for a variety of learning tasks, however deep nets typically exhibit forgetting when learning multiple tasks in sequence. To mitigate forgetting, we propose an experience replay process that augments the standard FIFO buffer and selectively stores experiences in a long-term memory. We explore four strategies for selecting which experiences will be stored: favoring surprise, favoring reward, matching the global training distribution, and maximizing coverage of the state space. We show that distribution matching successfully prevents catastrophic forgetting, and is consistently the best approach on all domains tested. While distribution matching has better and more consistent performance, we identify one case in which coverage maximization is beneficial - when tasks that receive less trained are more important. Overall, our results show that selective experience replay, when suitable selection algorithms are employed, can prevent catastrophic forgetting.
Tasks
Published	2018-02-28
URL	http://arxiv.org/abs/1802.10269v1
PDF	http://arxiv.org/pdf/1802.10269v1.pdf
PWC	https://paperswithcode.com/paper/selective-experience-replay-for-lifelong
Repo
Framework

Revisiting Salient Object Detection: Simultaneous Detection, Ranking, and Subitizing of Multiple Salient Objects


Title	Revisiting Salient Object Detection: Simultaneous Detection, Ranking, and Subitizing of Multiple Salient Objects
Authors	Md Amirul Islam, Mahmoud Kalash, Neil D. B. Bruce
Abstract	Salient object detection is a problem that has been considered in detail and many solutions proposed. In this paper, we argue that work to date has addressed a problem that is relatively ill-posed. Specifically, there is not universal agreement about what constitutes a salient object when multiple observers are queried. This implies that some objects are more likely to be judged salient than others, and implies a relative rank exists on salient objects. The solution presented in this paper solves this more general problem that considers relative rank, and we propose data and metrics suitable to measuring success in a relative object saliency landscape. A novel deep learning solution is proposed based on a hierarchical representation of relative saliency and stage-wise refinement. We also show that the problem of salient object subitizing can be addressed with the same network, and our approach exceeds performance of any prior work across all metrics considered (both traditional and newly proposed).
Tasks	Object Detection, Salient Object Detection
Published	2018-03-14
URL	http://arxiv.org/abs/1803.05082v2
PDF	http://arxiv.org/pdf/1803.05082v2.pdf
PWC	https://paperswithcode.com/paper/revisiting-salient-object-detection
Repo
Framework

Predicting Blood Pressure Response to Fluid Bolus Therapy Using Attention-Based Neural Networks for Clinical Interpretability


Title	Predicting Blood Pressure Response to Fluid Bolus Therapy Using Attention-Based Neural Networks for Clinical Interpretability
Authors	Uma M. Girkar, Ryo Uchimido, Li-wei H. Lehman, Peter Szolovits, Leo Celi, Wei-Hung Weng
Abstract	Determining whether hypotensive patients in intensive care units (ICUs) should receive fluid bolus therapy (FBT) has been an extremely challenging task for intensive care physicians as the corresponding increase in blood pressure has been hard to predict. Our study utilized regression models and attention-based recurrent neural network (RNN) algorithms and a multi-clinical information system large-scale database to build models that can predict the successful response to FBT among hypotensive patients in ICUs. We investigated both time-aggregated modeling using logistic regression algorithms with regularization and time-series modeling using the long short term memory network (LSTM) and the gated recurrent units network (GRU) with the attention mechanism for clinical interpretability. Among all modeling strategies, the stacked LSTM with the attention mechanism yielded the most predictable model with the highest accuracy of 0.852 and area under the curve (AUC) value of 0.925. The study results may help identify hypotensive patients in ICUs who will have sufficient blood pressure recovery after FBT.
Tasks	Time Series
Published	2018-12-03
URL	http://arxiv.org/abs/1812.00699v1
PDF	http://arxiv.org/pdf/1812.00699v1.pdf
PWC	https://paperswithcode.com/paper/predicting-blood-pressure-response-to-fluid
Repo
Framework

Performance Evaluation of SIFT Descriptor against Common Image Deformations on Iban Plaited Mat Motifs


Title	Performance Evaluation of SIFT Descriptor against Common Image Deformations on Iban Plaited Mat Motifs
Authors	Silvia Joseph, Irwandi Hipiny, Hamimah Ujir
Abstract	Borneo indigenous communities are blessed with rich craft heritage. One such examples is the Iban’s plaited mat craft. There have been many efforts by UNESCO and the Sarawak Government to preserve and promote the craft. One such method is by developing a mobile app capable of recognising the different mat motifs. As a first step towards this aim, we presents a novel image dataset consisting of seven mat motif classes. Each class possesses a unique variation of chevrons, diagonal shapes, symmetrical, repetitive, geometric and non geometric patterns. In this study, the performance of the Scale invariant feature transform (SIFT) descriptor is evaluated against five common image deformations, i.e., zoom and rotation, viewpoint, image blur, JPEG compression and illumination. Using our dataset, SIFT performed favourably with test sequences belonging to Illumination changes, Viewpoint changes, JPEG compression and Zoom and Rotation. However, it did not performed well with Image blur test sequences with an average of 1.61 percents retained pairwise matching after blurring with a Gaussian kernel of 8.0 radius.
Tasks
Published	2018-10-03
URL	http://arxiv.org/abs/1810.01562v1
PDF	http://arxiv.org/pdf/1810.01562v1.pdf
PWC	https://paperswithcode.com/paper/performance-evaluation-of-sift-descriptor
Repo
Framework

Boosted Density Estimation Remastered


Title	Boosted Density Estimation Remastered
Authors	Zac Cranko, Richard Nock
Abstract	There has recently been a steady increase in the number iterative approaches to density estimation. However, an accompanying burst of formal convergence guarantees has not followed; all results pay the price of heavy assumptions which are often unrealistic or hard to check. The Generative Adversarial Network (GAN) literature — seemingly orthogonal to the aforementioned pursuit — has had the side effect of a renewed interest in variational divergence minimisation (notably $f$-GAN). We show that by introducing a weak learning assumption (in the sense of the classical boosting framework) we are able to import some recent results from the GAN literature to develop an iterative boosted density estimation algorithm, including formal convergence results with rates, that does not suffer the shortcomings other approaches. We show that the density fit is an exponential family, and as part of our analysis obtain an improved variational characterisation of $f$-GAN.
Tasks	Density Estimation
Published	2018-03-22
URL	http://arxiv.org/abs/1803.08178v3
PDF	http://arxiv.org/pdf/1803.08178v3.pdf
PWC	https://paperswithcode.com/paper/boosted-density-estimation-remastered
Repo
Framework

Pooling homogeneous ensembles to build heterogeneous ones


Title	Pooling homogeneous ensembles to build heterogeneous ones
Authors	Maryam Sabzevari, Gonzalo Martínez-Muñoz, Alberto Suárez
Abstract	In ensemble methods, the outputs of a collection of diverse classifiers are combined in the expectation that the global prediction be more accurate than the individual ones. Heterogeneous ensembles consist of predictors of different types, which are likely to have different biases. If these biases are complementary, the combination of their decisions is beneficial. In this work, a family of heterogeneous ensembles is built by pooling classifiers from M homogeneous ensembles of different types of size T. Depending on the fraction of base classifiers of each type, a particular heterogeneous combination in this family is represented by a point in a regular simplex in M dimensions. The M vertices of this simplex represent the different homogeneous ensembles. A displacement away from one of these vertices effects a smooth transformation of the corresponding homogeneous ensemble into a heterogeneous one. The optimal composition of such heterogeneous ensemble can be determined using cross-validation or, if bootstrap samples are used to build the individual classifiers, out-of-bag data. An empirical analysis of such combinations of bootstraped ensembles composed of neural networks, SVMs, and random trees (i.e. from a standard random forest) illustrates the gains that can be achieved by this heterogeneous ensemble creation method.
Tasks
Published	2018-02-21
URL	https://arxiv.org/abs/1802.07877v2
PDF	https://arxiv.org/pdf/1802.07877v2.pdf
PWC	https://paperswithcode.com/paper/pooling-homogeneous-ensembles-to-build
Repo
Framework

Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference


Title	Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference
Authors	Louis C. Tiao, Edwin V. Bonilla, Fabio Ramos
Abstract	We formalize the problem of learning interdomain correspondences in the absence of paired data as Bayesian inference in a latent variable model (LVM), where one seeks the underlying hidden representations of entities from one domain as entities from the other domain. First, we introduce implicit latent variable models, where the prior over hidden representations can be specified flexibly as an implicit distribution. Next, we develop a new variational inference (VI) algorithm for this model based on minimization of the symmetric Kullback-Leibler (KL) divergence between a variational joint and the exact joint distribution. Lastly, we demonstrate that the state-of-the-art cycle-consistent adversarial learning (CYCLEGAN) models can be derived as a special case within our proposed VI framework, thus establishing its connection to approximate Bayesian inference methods.
Tasks	Bayesian Inference, Latent Variable Models
Published	2018-06-05
URL	http://arxiv.org/abs/1806.01771v3
PDF	http://arxiv.org/pdf/1806.01771v3.pdf
PWC	https://paperswithcode.com/paper/cycle-consistent-adversarial-learning-as
Repo
Framework