July 27, 2019

2999 words 15 mins read

Paper Group ANR 515

Dynamic Regret of Strongly Adaptive Methods. ScatterNet Hybrid Deep Learning (SHDL) Network For Object Classification. Online Boosting Algorithms for Multi-label Ranking. Emergence of Selective Invariance in Hierarchical Feed Forward Networks. A Change-Detection based Framework for Piecewise-stationary Multi-Armed Bandit Problem. Deciding How to De …

Dynamic Regret of Strongly Adaptive Methods


Title	Dynamic Regret of Strongly Adaptive Methods
Authors	Lijun Zhang, Tianbao Yang, Rong Jin, Zhi-Hua Zhou
Abstract	To cope with changing environments, recent developments in online learning have introduced the concepts of adaptive regret and dynamic regret independently. In this paper, we illustrate an intrinsic connection between these two concepts by showing that the dynamic regret can be expressed in terms of the adaptive regret and the functional variation. This observation implies that strongly adaptive algorithms can be directly leveraged to minimize the dynamic regret. As a result, we present a series of strongly adaptive algorithms that have small dynamic regrets for convex functions, exponentially concave functions, and strongly convex functions, respectively. To the best of our knowledge, this is the first time that exponential concavity is utilized to upper bound the dynamic regret. Moreover, all of those adaptive algorithms do not need any prior knowledge of the functional variation, which is a significant advantage over previous specialized methods for minimizing dynamic regret.
Tasks
Published	2017-01-26
URL	http://arxiv.org/abs/1701.07570v3
PDF	http://arxiv.org/pdf/1701.07570v3.pdf
PWC	https://paperswithcode.com/paper/dynamic-regret-of-strongly-adaptive-methods
Repo
Framework

ScatterNet Hybrid Deep Learning (SHDL) Network For Object Classification


Title	ScatterNet Hybrid Deep Learning (SHDL) Network For Object Classification
Authors	Amarjot Singh, Nick Kingsbury
Abstract	The paper proposes the ScatterNet Hybrid Deep Learning (SHDL) network that extracts invariant and discriminative image representations for object recognition. SHDL framework is constructed with a multi-layer ScatterNet front-end, an unsupervised learning middle, and a supervised learning back-end module. Each layer of the SHDL network is automatically designed as an explicit optimization problem leading to an optimal deep learning architecture with improved computational performance as compared to the more usual deep network architectures. SHDL network produces the state-of-the-art classification performance against unsupervised and semi-supervised learning (GANs) on two image datasets. Advantages of the SHDL network over supervised methods (NIN, VGG) are also demonstrated with experiments performed on training datasets of reduced size.
Tasks	Object Classification, Object Recognition
Published	2017-08-30
URL	http://arxiv.org/abs/1708.09212v1
PDF	http://arxiv.org/pdf/1708.09212v1.pdf
PWC	https://paperswithcode.com/paper/scatternet-hybrid-deep-learning-shdl-network
Repo
Framework

Online Boosting Algorithms for Multi-label Ranking


Title	Online Boosting Algorithms for Multi-label Ranking
Authors	Young Hun Jung, Ambuj Tewari
Abstract	We consider the multi-label ranking approach to multi-label learning. Boosting is a natural method for multi-label ranking as it aggregates weak predictions through majority votes, which can be directly used as scores to produce a ranking of the labels. We design online boosting algorithms with provable loss bounds for multi-label ranking. We show that our first algorithm is optimal in terms of the number of learners required to attain a desired accuracy, but it requires knowledge of the edge of the weak learners. We also design an adaptive algorithm that does not require this knowledge and is hence more practical. Experimental results on real data sets demonstrate that our algorithms are at least as good as existing batch boosting algorithms.
Tasks	Multi-Label Learning
Published	2017-10-23
URL	http://arxiv.org/abs/1710.08079v2
PDF	http://arxiv.org/pdf/1710.08079v2.pdf
PWC	https://paperswithcode.com/paper/online-boosting-algorithms-for-multi-label
Repo
Framework

Emergence of Selective Invariance in Hierarchical Feed Forward Networks


Title	Emergence of Selective Invariance in Hierarchical Feed Forward Networks
Authors	Dipan K. Pal, Vishnu Boddeti, Marios Savvides
Abstract	Many theories have emerged which investigate how in- variance is generated in hierarchical networks through sim- ple schemes such as max and mean pooling. The restriction to max/mean pooling in theoretical and empirical studies has diverted attention away from a more general way of generating invariance to nuisance transformations. We con- jecture that hierarchically building selective invariance (i.e. carefully choosing the range of the transformation to be in- variant to at each layer of a hierarchical network) is im- portant for pattern recognition. We utilize a novel pooling layer called adaptive pooling to find linear pooling weights within networks. These networks with the learnt pooling weights have performances on object categorization tasks that are comparable to max/mean pooling networks. In- terestingly, adaptive pooling can converge to mean pooling (when initialized with random pooling weights), find more general linear pooling schemes or even decide not to pool at all. We illustrate the general notion of selective invari- ance through object categorization experiments on large- scale datasets such as SVHN and ILSVRC 2012.
Tasks
Published	2017-01-30
URL	http://arxiv.org/abs/1701.08837v1
PDF	http://arxiv.org/pdf/1701.08837v1.pdf
PWC	https://paperswithcode.com/paper/emergence-of-selective-invariance-in
Repo
Framework

A Change-Detection based Framework for Piecewise-stationary Multi-Armed Bandit Problem


Title	A Change-Detection based Framework for Piecewise-stationary Multi-Armed Bandit Problem
Authors	Fang Liu, Joohyun Lee, Ness Shroff
Abstract	The multi-armed bandit problem has been extensively studied under the stationary assumption. However in reality, this assumption often does not hold because the distributions of rewards themselves may change over time. In this paper, we propose a change-detection (CD) based framework for multi-armed bandit problems under the piecewise-stationary setting, and study a class of change-detection based UCB (Upper Confidence Bound) policies, CD-UCB, that actively detects change points and restarts the UCB indices. We then develop CUSUM-UCB and PHT-UCB, that belong to the CD-UCB class and use cumulative sum (CUSUM) and Page-Hinkley Test (PHT) to detect changes. We show that CUSUM-UCB obtains the best known regret upper bound under mild assumptions. We also demonstrate the regret reduction of the CD-UCB policies over arbitrary Bernoulli rewards and Yahoo! datasets of webpage click-through rates.
Tasks
Published	2017-11-08
URL	http://arxiv.org/abs/1711.03539v2
PDF	http://arxiv.org/pdf/1711.03539v2.pdf
PWC	https://paperswithcode.com/paper/a-change-detection-based-framework-for
Repo
Framework

Deciding How to Decide: Dynamic Routing in Artificial Neural Networks


Title	Deciding How to Decide: Dynamic Routing in Artificial Neural Networks
Authors	Mason McGill, Pietro Perona
Abstract	We propose and systematically evaluate three strategies for training dynamically-routed artificial neural networks: graphs of learned transformations through which different input signals may take different paths. Though some approaches have advantages over others, the resulting networks are often qualitatively similar. We find that, in dynamically-routed networks trained to classify images, layers and branches become specialized to process distinct categories of images. Additionally, given a fixed computational budget, dynamically-routed networks tend to perform better than comparable statically-routed networks.
Tasks
Published	2017-03-17
URL	http://arxiv.org/abs/1703.06217v2
PDF	http://arxiv.org/pdf/1703.06217v2.pdf
PWC	https://paperswithcode.com/paper/deciding-how-to-decide-dynamic-routing-in
Repo
Framework

Reconstruction of the External Stimuli from Brain Signals


Title	Reconstruction of the External Stimuli from Brain Signals
Authors	Pouya Ghaemmaghami
Abstract	Despite the rapid advances in Brain-computer Interfacing (BCI) and continuous effort to improve the accuracy of brain decoding systems, the urge for the systems to reconstruct the experiences of the users has been widely acknowledged. This urge has been investigated by some researchers during the past years in terms of reconstruction of the naturalistic images, abstract images, video and audio. In this study, we try to tackle this issue by regressing the stimuli spectrogram using the spectrogram analysis of the brain signals. The results of our regression-based method suggest the feasibility of such reconstructions using the neuroimaging techniques that are appropriate for out-of-lab scenarios.
Tasks	Brain Decoding
Published	2017-11-14
URL	http://arxiv.org/abs/1711.06550v1
PDF	http://arxiv.org/pdf/1711.06550v1.pdf
PWC	https://paperswithcode.com/paper/reconstruction-of-the-external-stimuli-from
Repo
Framework

Training-image based geostatistical inversion using a spatial generative adversarial neural network


Title	Training-image based geostatistical inversion using a spatial generative adversarial neural network
Authors	Eric Laloy, Romain Hérault, Diederik Jacques, Niklas Linde
Abstract	Probabilistic inversion within a multiple-point statistics framework is often computationally prohibitive for high-dimensional problems. To partly address this, we introduce and evaluate a new training-image based inversion approach for complex geologic media. Our approach relies on a deep neural network of the generative adversarial network (GAN) type. After training using a training image (TI), our proposed spatial GAN (SGAN) can quickly generate 2D and 3D unconditional realizations. A key characteristic of our SGAN is that it defines a (very) low-dimensional parameterization, thereby allowing for efficient probabilistic inversion using state-of-the-art Markov chain Monte Carlo (MCMC) methods. In addition, available direct conditioning data can be incorporated within the inversion. Several 2D and 3D categorical TIs are first used to analyze the performance of our SGAN for unconditional geostatistical simulation. Training our deep network can take several hours. After training, realizations containing a few millions of pixels/voxels can be produced in a matter of seconds. This makes it especially useful for simulating many thousands of realizations (e.g., for MCMC inversion) as the relative cost of the training per realization diminishes with the considered number of realizations. Synthetic inversion case studies involving 2D steady-state flow and 3D transient hydraulic tomography with and without direct conditioning data are used to illustrate the effectiveness of our proposed SGAN-based inversion. For the 2D case, the inversion rapidly explores the posterior model distribution. For the 3D case, the inversion recovers model realizations that fit the data close to the target level and visually resemble the true model well.
Tasks
Published	2017-08-16
URL	http://arxiv.org/abs/1708.04975v2
PDF	http://arxiv.org/pdf/1708.04975v2.pdf
PWC	https://paperswithcode.com/paper/training-image-based-geostatistical-inversion
Repo
Framework

Face Sketch Matching via Coupled Deep Transform Learning


Title	Face Sketch Matching via Coupled Deep Transform Learning
Authors	Shruti Nagpal, Maneet Singh, Richa Singh, Mayank Vatsa, Afzel Noore, Angshul Majumdar
Abstract	Face sketch to digital image matching is an important challenge of face recognition that involves matching across different domains. Current research efforts have primarily focused on extracting domain invariant representations or learning a mapping from one domain to the other. In this research, we propose a novel transform learning based approach termed as DeepTransformer, which learns a transformation and mapping function between the features of two domains. The proposed formulation is independent of the input information and can be applied with any existing learned or hand-crafted feature. Since the mapping function is directional in nature, we propose two variants of DeepTransformer: (i) semi-coupled and (ii) symmetrically-coupled deep transform learning. This research also uses a novel IIIT-D Composite Sketch with Age (CSA) variations database which contains sketch images of 150 subjects along with age-separated digital photos. The performance of the proposed models is evaluated on a novel application of sketch-to-sketch matching, along with sketch-to-digital photo matching. Experimental results demonstrate the robustness of the proposed models in comparison to existing state-of-the-art sketch matching algorithms and a commercial face recognition system.
Tasks	Face Recognition
Published	2017-10-09
URL	http://arxiv.org/abs/1710.02914v1
PDF	http://arxiv.org/pdf/1710.02914v1.pdf
PWC	https://paperswithcode.com/paper/face-sketch-matching-via-coupled-deep
Repo
Framework

Quantifying Performance of Bipedal Standing with Multi-channel EMG


Title	Quantifying Performance of Bipedal Standing with Multi-channel EMG
Authors	Yanan Sui, Kun ho Kim, Joel W. Burdick
Abstract	Spinal cord stimulation has enabled humans with motor complete spinal cord injury (SCI) to independently stand and recover some lost autonomic function. Quantifying the quality of bipedal standing under spinal stimulation is important for spinal rehabilitation therapies and for new strategies that seek to combine spinal stimulation and rehabilitative robots (such as exoskeletons) in real time feedback. To study the potential for automated electromyography (EMG) analysis in SCI, we evaluated the standing quality of paralyzed patients undergoing electrical spinal cord stimulation using both video and multi-channel surface EMG recordings during spinal stimulation therapy sessions. The quality of standing under different stimulation settings was quantified manually by experienced clinicians. By correlating features of the recorded EMG activity with the expert evaluations, we show that multi-channel EMG recording can provide accurate, fast, and robust estimation for the quality of bipedal standing in spinally stimulated SCI patients. Moreover, our analysis shows that the total number of EMG channels needed to effectively predict standing quality can be reduced while maintaining high estimation accuracy, which provides more flexibility for rehabilitation robotic systems to incorporate EMG recordings.
Tasks	Electromyography (EMG)
Published	2017-11-21
URL	http://arxiv.org/abs/1711.07894v1
PDF	http://arxiv.org/pdf/1711.07894v1.pdf
PWC	https://paperswithcode.com/paper/quantifying-performance-of-bipedal-standing
Repo
Framework

Additivity of Information in Multilayer Networks via Additive Gaussian Noise Transforms


Title	Additivity of Information in Multilayer Networks via Additive Gaussian Noise Transforms
Authors	Galen Reeves
Abstract	Multilayer (or deep) networks are powerful probabilistic models based on multiple stages of a linear transform followed by a non-linear (possibly random) function. In general, the linear transforms are defined by matrices and the non-linear functions are defined by information channels. These models have gained great popularity due to their ability to characterize complex probabilistic relationships arising in a wide variety of inference problems. The contribution of this paper is a new method for analyzing the fundamental limits of statistical inference in settings where the model is known. The validity of our method can be established in a number of settings and is conjectured to hold more generally. A key assumption made throughout is that the matrices are drawn randomly from orthogonally invariant distributions. Our method yields explicit formulas for 1) the mutual information; 2) the minimum mean-squared error (MMSE); 3) the existence and locations of certain phase-transitions with respect to the problem parameters; and 4) the stationary points for the state evolution of approximate message passing algorithms. When applied to the special case of models with multivariate Gaussian channels our method is rigorous and has close connections to free probability theory for random matrices. When applied to the general case of non-Gaussian channels, our method provides a simple alternative to the replica method from statistical physics. A key observation is that the combined effects of the individual components in the model (namely the matrices and the channels) are additive when viewed in a certain transform domain.
Tasks
Published	2017-10-12
URL	http://arxiv.org/abs/1710.04580v1
PDF	http://arxiv.org/pdf/1710.04580v1.pdf
PWC	https://paperswithcode.com/paper/additivity-of-information-in-multilayer
Repo
Framework

Streaming Binary Sketching based on Subspace Tracking and Diagonal Uniformization


Title	Streaming Binary Sketching based on Subspace Tracking and Diagonal Uniformization
Authors	Anne Morvan, Antoine Souloumiac, Cédric Gouy-Pailler, Jamal Atif
Abstract	In this paper, we address the problem of learning compact similarity-preserving embeddings for massive high-dimensional streams of data in order to perform efficient similarity search. We present a new online method for computing binary compressed representations -sketches- of high-dimensional real feature vectors. Given an expected code length $c$ and high-dimensional input data points, our algorithm provides a $c$-bits binary code for preserving the distance between the points from the original high-dimensional space. Our algorithm does not require neither the storage of the whole dataset nor a chunk, thus it is fully adaptable to the streaming setting. It also provides low time complexity and convergence guarantees. We demonstrate the quality of our binary sketches through experiments on real data for the nearest neighbors search task in the online setting.
Tasks
Published	2017-05-22
URL	http://arxiv.org/abs/1705.07661v2
PDF	http://arxiv.org/pdf/1705.07661v2.pdf
PWC	https://paperswithcode.com/paper/streaming-binary-sketching-based-on-subspace
Repo
Framework

A Knowledge-Based Approach to Word Sense Disambiguation by distributional selection and semantic features


Title	A Knowledge-Based Approach to Word Sense Disambiguation by distributional selection and semantic features
Authors	Mokhtar Billami
Abstract	Word sense disambiguation improves many Natural Language Processing (NLP) applications such as Information Retrieval, Information Extraction, Machine Translation, or Lexical Simplification. Roughly speaking, the aim is to choose for each word in a text its best sense. One of the most popular method estimates local semantic similarity relatedness between two word senses and then extends it to all words from text. The most direct method computes a rough score for every pair of word senses and chooses the lexical chain that has the best score (we can imagine the exponential complexity that returns this comprehensive approach). In this paper, we propose to use a combinatorial optimization metaheuristic for choosing the nearest neighbors obtained by distributional selection around the word to disambiguate. The test and the evaluation of our method concern a corpus written in French by means of the semantic network BabelNet. The obtained accuracy rate is 78 % on all names and verbs chosen for the evaluation.
Tasks	Combinatorial Optimization, Information Retrieval, Lexical Simplification, Machine Translation, Semantic Similarity, Semantic Textual Similarity, Word Sense Disambiguation
Published	2017-02-27
URL	http://arxiv.org/abs/1702.08450v1
PDF	http://arxiv.org/pdf/1702.08450v1.pdf
PWC	https://paperswithcode.com/paper/a-knowledge-based-approach-to-word-sense
Repo
Framework

Self-Supervised Learning for Spinal MRIs


Title	Self-Supervised Learning for Spinal MRIs
Authors	Amir Jamaludin, Timor Kadir, Andrew Zisserman
Abstract	A significant proportion of patients scanned in a clinical setting have follow-up scans. We show in this work that such longitudinal scans alone can be used as a form of ‘free’ self-supervision for training a deep network. We demonstrate this self-supervised learning for the case of T2-weighted sagittal lumbar Magnetic Resonance Images (MRIs). A Siamese convolutional neural network (CNN) is trained using two losses: (i) a contrastive loss on whether the scan is of the same person (i.e. longitudinal) or not, together with (ii) a classification loss on predicting the level of vertebral bodies. The performance of this pre-trained network is then assessed on a grading classification task. We experiment on a dataset of 1016 subjects, 423 possessing follow-up scans, with the end goal of learning the disc degeneration radiological gradings attached to the intervertebral discs. We show that the performance of the pre-trained CNN on the supervised classification task is (i) superior to that of a network trained from scratch; and (ii) requires far fewer annotated training samples to reach an equivalent performance to that of the network trained from scratch.
Tasks
Published	2017-08-01
URL	http://arxiv.org/abs/1708.00367v1
PDF	http://arxiv.org/pdf/1708.00367v1.pdf
PWC	https://paperswithcode.com/paper/self-supervised-learning-for-spinal-mris
Repo
Framework

A New Unbiased and Efficient Class of LSH-Based Samplers and Estimators for Partition Function Computation in Log-Linear Models


Title	A New Unbiased and Efficient Class of LSH-Based Samplers and Estimators for Partition Function Computation in Log-Linear Models
Authors	Ryan Spring, Anshumali Shrivastava
Abstract	Log-linear models are arguably the most successful class of graphical models for large-scale applications because of their simplicity and tractability. Learning and inference with these models require calculating the partition function, which is a major bottleneck and intractable for large state spaces. Importance Sampling (IS) and MCMC-based approaches are lucrative. However, the condition of having a “good” proposal distribution is often not satisfied in practice. In this paper, we add a new dimension to efficient estimation via sampling. We propose a new sampling scheme and an unbiased estimator that estimates the partition function accurately in sub-linear time. Our samples are generated in near-constant time using locality sensitive hashing (LSH), and so are correlated and unnormalized. We demonstrate the effectiveness of our proposed approach by comparing the accuracy and speed of estimating the partition function against other state-of-the-art estimation techniques including IS and the efficient variant of Gumbel-Max sampling. With our efficient sampling scheme, we accurately train real-world language models using only 1-2% of computations.
Tasks
Published	2017-03-15
URL	http://arxiv.org/abs/1703.05160v1
PDF	http://arxiv.org/pdf/1703.05160v1.pdf
PWC	https://paperswithcode.com/paper/a-new-unbiased-and-efficient-class-of-lsh
Repo
Framework