May 7, 2019

3319 words 16 mins read

Paper Group ANR 91

A Video-Based Method for Objectively Rating Ataxia. An optimal learning method for developing personalized treatment regimes. Topology and Geometry of Half-Rectified Network Optimization. Information Dropout: Learning Optimal Representations Through Noisy Computation. Watch-n-Patch: Unsupervised Learning of Actions and Relations. Revise Saturated A …

A Video-Based Method for Objectively Rating Ataxia


Title	A Video-Based Method for Objectively Rating Ataxia
Authors	Ronnachai Jaroensri, Amy Zhao, Guha Balakrishnan, Derek Lo, Jeremy Schmahmann, John Guttag, Fredo Durand
Abstract	For many movement disorders, such as Parkinson’s disease and ataxia, disease progression is visually assessed by a clinician using a numerical disease rating scale. These tests are subjective, time-consuming, and must be administered by a professional. This can be problematic where specialists are not available, or when a patient is not consistently evaluated by the same clinician. We present an automated method for quantifying the severity of motion impairment in patients with ataxia, using only video recordings. We consider videos of the finger-to-nose test, a common movement task used as part of the assessment of ataxia progression during the course of routine clinical checkups. Our method uses neural network-based pose estimation and optical flow techniques to track the motion of the patient’s hand in a video recording. We extract features that describe qualities of the motion such as speed and variation in performance. Using labels provided by an expert clinician, we train a supervised learning model that predicts severity according to the Brief Ataxia Rating Scale (BARS). The performance of our system is comparable to that of a group of ataxia specialists in terms of mean error and correlation, and our system’s predictions were consistently within the range of inter-rater variability. This work demonstrates the feasibility of using computer vision and machine learning to produce consistent and clinically useful measures of motor impairment.
Tasks	Optical Flow Estimation, Pose Estimation
Published	2016-12-13
URL	http://arxiv.org/abs/1612.04007v3
PDF	http://arxiv.org/pdf/1612.04007v3.pdf
PWC	https://paperswithcode.com/paper/a-video-based-method-for-objectively-rating
Repo
Framework

An optimal learning method for developing personalized treatment regimes


Title	An optimal learning method for developing personalized treatment regimes
Authors	Yingfei Wang, Warren Powell
Abstract	A treatment regime is a function that maps individual patient information to a recommended treatment, hence explicitly incorporating the heterogeneity in need for treatment across individuals. Patient responses are dichotomous and can be predicted through an unknown relationship that depends on the patient information and the selected treatment. The goal is to find the treatments that lead to the best patient responses on average. Each experiment is expensive, forcing us to learn the most from each experiment. We adopt a Bayesian approach both to incorporate possible prior information and to update our treatment regime continuously as information accrues, with the potential to allow smaller yet more informative trials and for patients to receive better treatment. By formulating the problem as contextual bandits, we introduce a knowledge gradient policy to guide the treatment assignment by maximizing the expected value of information, for which an approximation method is used to overcome computational challenges. We provide a detailed study on how to make sequential medical decisions under uncertainty to reduce health care costs on a real world knee replacement dataset. We use clustering and LASSO to deal with the intrinsic sparsity in health datasets. We show experimentally that even though the problem is sparse, through careful selection of physicians (versus picking them at random), we can significantly improve the success rates.
Tasks	Multi-Armed Bandits
Published	2016-07-06
URL	http://arxiv.org/abs/1607.01462v1
PDF	http://arxiv.org/pdf/1607.01462v1.pdf
PWC	https://paperswithcode.com/paper/an-optimal-learning-method-for-developing
Repo
Framework

Topology and Geometry of Half-Rectified Network Optimization


Title	Topology and Geometry of Half-Rectified Network Optimization
Authors	C. Daniel Freeman, Joan Bruna
Abstract	The loss surface of deep neural networks has recently attracted interest in the optimization and machine learning communities as a prime example of high-dimensional non-convex problem. Some insights were recently gained using spin glass models and mean-field approximations, but at the expense of strongly simplifying the nonlinear nature of the model. In this work, we do not make any such assumption and study conditions on the data distribution and model architecture that prevent the existence of bad local minima. Our theoretical work quantifies and formalizes two important \emph{folklore} facts: (i) the landscape of deep linear networks has a radically different topology from that of deep half-rectified ones, and (ii) that the energy landscape in the non-linear case is fundamentally controlled by the interplay between the smoothness of the data distribution and model over-parametrization. Our main theoretical contribution is to prove that half-rectified single layer networks are asymptotically connected, and we provide explicit bounds that reveal the aforementioned interplay. The conditioning of gradient descent is the next challenge we address. We study this question through the geometry of the level sets, and we introduce an algorithm to efficiently estimate the regularity of such sets on large-scale networks. Our empirical results show that these level sets remain connected throughout all the learning phase, suggesting a near convex behavior, but they become exponentially more curvy as the energy level decays, in accordance to what is observed in practice with very low curvature attractors.
Tasks
Published	2016-11-04
URL	http://arxiv.org/abs/1611.01540v4
PDF	http://arxiv.org/pdf/1611.01540v4.pdf
PWC	https://paperswithcode.com/paper/topology-and-geometry-of-half-rectified
Repo
Framework

Information Dropout: Learning Optimal Representations Through Noisy Computation


Title	Information Dropout: Learning Optimal Representations Through Noisy Computation
Authors	Alessandro Achille, Stefano Soatto
Abstract	The cross-entropy loss commonly used in deep learning is closely related to the defining properties of optimal representations, but does not enforce some of the key properties. We show that this can be solved by adding a regularization term, which is in turn related to injecting multiplicative noise in the activations of a Deep Neural Network, a special case of which is the common practice of dropout. We show that our regularized loss function can be efficiently minimized using Information Dropout, a generalization of dropout rooted in information theoretic principles that automatically adapts to the data and can better exploit architectures of limited capacity. When the task is the reconstruction of the input, we show that our loss function yields a Variational Autoencoder as a special case, thus providing a link between representation learning, information theory and variational inference. Finally, we prove that we can promote the creation of disentangled representations simply by enforcing a factorized prior, a fact that has been observed empirically in recent work. Our experiments validate the theoretical intuitions behind our method, and we find that information dropout achieves a comparable or better generalization performance than binary dropout, especially on smaller models, since it can automatically adapt the noise to the structure of the network, as well as to the test sample.
Tasks	Representation Learning
Published	2016-11-04
URL	http://arxiv.org/abs/1611.01353v3
PDF	http://arxiv.org/pdf/1611.01353v3.pdf
PWC	https://paperswithcode.com/paper/information-dropout-learning-optimal
Repo
Framework

Watch-n-Patch: Unsupervised Learning of Actions and Relations


Title	Watch-n-Patch: Unsupervised Learning of Actions and Relations
Authors	Chenxia Wu, Jiemi Zhang, Ozan Sener, Bart Selman, Silvio Savarese, Ashutosh Saxena
Abstract	There is a large variation in the activities that humans perform in their everyday lives. We consider modeling these composite human activities which comprises multiple basic level actions in a completely unsupervised setting. Our model learns high-level co-occurrence and temporal relations between the actions. We consider the video as a sequence of short-term action clips, which contains human-words and object-words. An activity is about a set of action-topics and object-topics indicating which actions are present and which objects are interacting with. We then propose a new probabilistic model relating the words and the topics. It allows us to model long-range action relations that commonly exist in the composite activities, which is challenging in previous works. We apply our model to the unsupervised action segmentation and clustering, and to a novel application that detects forgotten actions, which we call action patching. For evaluation, we contribute a new challenging RGB-D activity video dataset recorded by the new Kinect v2, which contains several human daily activities as compositions of multiple actions interacting with different objects. Moreover, we develop a robotic system that watches people and reminds people by applying our action patching algorithm. Our robotic setup can be easily deployed on any assistive robot.
Tasks	action segmentation
Published	2016-03-11
URL	http://arxiv.org/abs/1603.03541v1
PDF	http://arxiv.org/pdf/1603.03541v1.pdf
PWC	https://paperswithcode.com/paper/watch-n-patch-unsupervised-learning-of
Repo
Framework

Revise Saturated Activation Functions


Title	Revise Saturated Activation Functions
Authors	Bing Xu, Ruitong Huang, Mu Li
Abstract	In this paper, we revise two commonly used saturated functions, the logistic sigmoid and the hyperbolic tangent (tanh). We point out that, besides the well-known non-zero centered property, slope of the activation function near the origin is another possible reason making training deep networks with the logistic function difficult to train. We demonstrate that, with proper rescaling, the logistic sigmoid achieves comparable results with tanh. Then following the same argument, we improve tahn by penalizing in the negative part. We show that “penalized tanh” is comparable and even outperforms the state-of-the-art non-saturated functions including ReLU and leaky ReLU on deep convolution neural networks. Our results contradict to the conclusion of previous works that the saturation property causes the slow convergence. It suggests further investigation is necessary to better understand activation functions in deep architectures.
Tasks
Published	2016-02-18
URL	http://arxiv.org/abs/1602.05980v2
PDF	http://arxiv.org/pdf/1602.05980v2.pdf
PWC	https://paperswithcode.com/paper/revise-saturated-activation-functions
Repo
Framework

The Schema Editor of OpenIoT for Semantic Sensor Networks


Title	The Schema Editor of OpenIoT for Semantic Sensor Networks
Authors	Prem Prakash Jayaraman, Jean-Paul Calbimonte, Hoan Nguyen Mau Quoc
Abstract	Ontologies provide conceptual abstractions over data, in domains such as the Internet of Things, in a way that sensor data can be harvested and interpreted by people and applications. The Semantic Sensor Network (SSN) ontology is the de-facto standard for semantic representation of sensor observations and metadata, and it is used at the core of the open source platform for the Internet of Things, OpenIoT. In this paper we present a Schema Editor that provides an intuitive web interface for defining new types of sensors, and concrete instances of them, using the SSN ontology as the core model. This editor is fully integrated with the OpenIoT platform for generating virtual sensor descriptions and automating their semantic annotation and registration process.
Tasks
Published	2016-06-21
URL	http://arxiv.org/abs/1606.06434v1
PDF	http://arxiv.org/pdf/1606.06434v1.pdf
PWC	https://paperswithcode.com/paper/the-schema-editor-of-openiot-for-semantic
Repo
Framework

On the representation and embedding of knowledge bases beyond binary relations


Title	On the representation and embedding of knowledge bases beyond binary relations
Authors	Jianfeng Wen, Jianxin Li, Yongyi Mao, Shini Chen, Richong Zhang
Abstract	The models developed to date for knowledge base embedding are all based on the assumption that the relations contained in knowledge bases are binary. For the training and testing of these embedding models, multi-fold (or n-ary) relational data are converted to triples (e.g., in FB15K dataset) and interpreted as instances of binary relations. This paper presents a canonical representation of knowledge bases containing multi-fold relations. We show that the existing embedding models on the popular FB15K datasets correspond to a sub-optimal modelling framework, resulting in a loss of structural information. We advocate a novel modelling framework, which models multi-fold relations directly using this canonical representation. Using this framework, the existing TransH model is generalized to a new model, m-TransH. We demonstrate experimentally that m-TransH outperforms TransH by a large margin, thereby establishing a new state of the art.
Tasks
Published	2016-04-28
URL	http://arxiv.org/abs/1604.08642v1
PDF	http://arxiv.org/pdf/1604.08642v1.pdf
PWC	https://paperswithcode.com/paper/on-the-representation-and-embedding-of
Repo
Framework

Fast moment estimation for generalized latent Dirichlet models


Title	Fast moment estimation for generalized latent Dirichlet models
Authors	Shiwen Zhao, Barbara E. Engelhardt, Sayan Mukherjee, David B. Dunson
Abstract	We develop a generalized method of moments (GMM) approach for fast parameter estimation in a new class of Dirichlet latent variable models with mixed data types. Parameter estimation via GMM has been demonstrated to have computational and statistical advantages over alternative methods, such as expectation maximization, variational inference, and Markov chain Monte Carlo. The key computational advan- tage of our method (MELD) is that parameter estimation does not require instantiation of the latent variables. Moreover, a representational advantage of the GMM approach is that the behavior of the model is agnostic to distributional assumptions of the observations. We derive population moment conditions after marginalizing out the sample-specific Dirichlet latent variables. The moment conditions only depend on component mean parameters. We illustrate the utility of our approach on simulated data, comparing results from MELD to alternative methods, and we show the promise of our approach through the application of MELD to several data sets.
Tasks	Latent Variable Models
Published	2016-03-17
URL	http://arxiv.org/abs/1603.05324v2
PDF	http://arxiv.org/pdf/1603.05324v2.pdf
PWC	https://paperswithcode.com/paper/fast-moment-estimation-for-generalized-latent
Repo
Framework

3D Display Calibration by Visual Pattern Analysis


Title	3D Display Calibration by Visual Pattern Analysis
Authors	Hyoseok Hwang, Hyun Sung Chang, Dongkyung Nam, In So Kweon
Abstract	Nearly all 3D displays need calibration for correct rendering. More often than not, the optical elements in a 3D display are misaligned from the designed parameter setting. As a result, 3D magic does not perform well as intended. The observed images tend to get distorted. In this paper, we propose a novel display calibration method to fix the situation. In our method, a pattern image is displayed on the panel and a camera takes its pictures twice at different positions. Then, based on a quantitative model, we extract all display parameters (i.e., pitch, slanted angle, gap or thickness, offset) from the observed patterns in the captured images. For high accuracy and robustness, our method analyzes the patterns mostly in frequency domain. We conduct two types of experiments for validation; one with optical simulation for quantitative results and the other with real-life displays for qualitative assessment. Experimental results demonstrate that our method is quite accurate, about a half order of magnitude higher than prior work; is efficient, spending less than 2 s for computation; and is robust to noise, working well in the SNR regime as low as 6 dB.
Tasks	Calibration
Published	2016-06-23
URL	http://arxiv.org/abs/1606.07166v1
PDF	http://arxiv.org/pdf/1606.07166v1.pdf
PWC	https://paperswithcode.com/paper/3d-display-calibration-by-visual-pattern
Repo
Framework

Visual Stability Prediction and Its Application to Manipulation


Title	Visual Stability Prediction and Its Application to Manipulation
Authors	Wenbin Li, Aleš Leonardis, Mario Fritz
Abstract	Understanding physical phenomena is a key competence that enables humans and animals to act and interact under uncertain perception in previously unseen environments containing novel objects and their configurations. Developmental psychology has shown that such skills are acquired by infants from observations at a very early stage. In this paper, we contrast a more traditional approach of taking a model-based route with explicit 3D representations and physical simulation by an {\em end-to-end} approach that directly predicts stability from appearance. We ask the question if and to what extent and quality such a skill can directly be acquired in a data-driven way—bypassing the need for an explicit simulation at run-time. We present a learning-based approach based on simulated data that predicts stability of towers comprised of wooden blocks under different conditions and quantities related to the potential fall of the towers. We first evaluate the approach on synthetic data and compared the results to human judgments on the same stimuli. Further, we extend this approach to reason about future states of such towers that in turn enables successful stacking.
Tasks
Published	2016-09-15
URL	http://arxiv.org/abs/1609.04861v2
PDF	http://arxiv.org/pdf/1609.04861v2.pdf
PWC	https://paperswithcode.com/paper/visual-stability-prediction-and-its
Repo
Framework

Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing


Title	Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
Authors	Steven K. Esser, Paul A. Merolla, John V. Arthur, Andrew S. Cassidy, Rathinakumar Appuswamy, Alexander Andreopoulos, David J. Berg, Jeffrey L. McKinstry, Timothy Melano, Davis R. Barch, Carmelo di Nolfo, Pallab Datta, Arnon Amir, Brian Taba, Myron D. Flickner, Dharmendra S. Modha
Abstract	Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that i) approach state-of-the-art classification accuracy across 8 standard datasets, encompassing vision and speech, ii) perform inference while preserving the hardware’s underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1200 and 2600 frames per second and using between 25 and 275 mW (effectively > 6000 frames / sec / W) and iii) can be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. For the first time, the algorithmic power of deep learning can be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer.
Tasks
Published	2016-03-28
URL	http://arxiv.org/abs/1603.08270v2
PDF	http://arxiv.org/pdf/1603.08270v2.pdf
PWC	https://paperswithcode.com/paper/convolutional-networks-for-fast-energy
Repo
Framework

Learning From Graph Neighborhoods Using LSTMs


Title	Learning From Graph Neighborhoods Using LSTMs
Authors	Rakshit Agrawal, Luca de Alfaro, Vassilis Polychronopoulos
Abstract	Many prediction problems can be phrased as inferences over local neighborhoods of graphs. The graph represents the interaction between entities, and the neighborhood of each entity contains information that allows the inferences or predictions. We present an approach for applying machine learning directly to such graph neighborhoods, yielding predicitons for graph nodes on the basis of the structure of their local neighborhood and the features of the nodes in it. Our approach allows predictions to be learned directly from examples, bypassing the step of creating and tuning an inference model or summarizing the neighborhoods via a fixed set of hand-crafted features. The approach is based on a multi-level architecture built from Long Short-Term Memory neural nets (LSTMs); the LSTMs learn how to summarize the neighborhood from data. We demonstrate the effectiveness of the proposed technique on a synthetic example and on real-world data related to crowdsourced grading, Bitcoin transactions, and Wikipedia edit reversions.
Tasks
Published	2016-11-21
URL	http://arxiv.org/abs/1611.06882v1
PDF	http://arxiv.org/pdf/1611.06882v1.pdf
PWC	https://paperswithcode.com/paper/learning-from-graph-neighborhoods-using-lstms
Repo
Framework

Modeling cumulative biological phenomena with Suppes-Bayes Causal Networks


Title	Modeling cumulative biological phenomena with Suppes-Bayes Causal Networks
Authors	Daniele Ramazzotti, Alex Graudenzi, Giulio Caravagna, Marco Antoniotti
Abstract	Several diseases related to cell proliferation are characterized by the accumulation of somatic DNA changes, with respect to wildtype conditions. Cancer and HIV are two common examples of such diseases, where the mutational load in the cancerous/viral population increases over time. In these cases, selective pressures are often observed along with competition, cooperation and parasitism among distinct cellular clones. Recently, we presented a mathematical framework to model these phenomena, based on a combination of Bayesian inference and Suppes’ theory of probabilistic causation, depicted in graphical structures dubbed Suppes-Bayes Causal Networks (SBCNs). SBCNs are generative probabilistic graphical models that recapitulate the potential ordering of accumulation of such DNA changes during the progression of the disease. Such models can be inferred from data by exploiting likelihood-based model-selection strategies with regularization. In this paper we discuss the theoretical foundations of our approach and we investigate in depth the influence on the model-selection task of: (i) the poset based on Suppes’ theory and (ii) different regularization strategies. Furthermore, we provide an example of application of our framework to HIV genetic data highlighting the valuable insights provided by the inferred.
Tasks	Bayesian Inference, Model Selection
Published	2016-02-25
URL	http://arxiv.org/abs/1602.07857v4
PDF	http://arxiv.org/pdf/1602.07857v4.pdf
PWC	https://paperswithcode.com/paper/modeling-cumulative-biological-phenomena-with
Repo
Framework

Improving Efficiency of SVM k-fold Cross-validation by Alpha Seeding


Title	Improving Efficiency of SVM k-fold Cross-validation by Alpha Seeding
Authors	Zeyi Wen, Bin Li, Rao Kotagiri, Jian Chen, Yawen Chen, Rui Zhang
Abstract	The k-fold cross-validation is commonly used to evaluate the effectiveness of SVMs with the selected hyper-parameters. It is known that the SVM k-fold cross-validation is expensive, since it requires training k SVMs. However, little work has explored reusing the h-th SVM for training the (h+1)-th SVM for improving the efficiency of k-fold cross-validation. In this paper, we propose three algorithms that reuse the h-th SVM for improving the efficiency of training the (h+1)-th SVM. Our key idea is to efficiently identify the support vectors and to accurately estimate their associated weights (also called alpha values) of the next SVM by using the previous SVM. Our experimental results show that our algorithms are several times faster than the k-fold cross-validation which does not make use of the previously trained SVM. Moreover, our algorithms produce the same results (hence same accuracy) as the k-fold cross-validation which does not make use of the previously trained SVM.
Tasks
Published	2016-11-23
URL	http://arxiv.org/abs/1611.07659v2
PDF	http://arxiv.org/pdf/1611.07659v2.pdf
PWC	https://paperswithcode.com/paper/improving-efficiency-of-svm-k-fold-cross
Repo
Framework