Paper Group ANR 85
Learning to Recognize Objects by Retaining other Factors of Variation. Learning to Extract Motion from Videos in Convolutional Neural Networks. DeepProposals: Hunting Objects and Actions by Cascading Deep Convolutional Layers. Optimal Rates for Multi-pass Stochastic Gradient Methods. FLAG n’ FLARE: Fast Linearly-Coupled Adaptive Gradient Methods. A …
Learning to Recognize Objects by Retaining other Factors of Variation
| Title | Learning to Recognize Objects by Retaining other Factors of Variation |
| Authors | Jiaping Zhao, Chin-kai Chang, Laurent Itti |
| Abstract | Natural images are generated under many factors, including shape, pose, illumination etc. Most existing ConvNets formulate object recognition from natural images as a single task classification problem, and attempt to learn features useful for object categories, but invariant to other factors of variation as much as possible. These architectures do not explicitly learn other factors, like pose and lighting, instead, they usually discard them by pooling and normalization. In this work, we take the opposite approach: we train ConvNets for object recognition by retaining other factors (pose in our case) and learn them jointly with object category. We design a new multi-task leaning (MTL) ConvNet, named disentangling CNN (disCNN), which explicitly enforces the disentangled representations of object identity and pose, and is trained to predict object categories and pose transformations. We show that disCNN achieves significantly better object recognition accuracies than AlexNet trained solely to predict object categories on the iLab-20M dataset, which is a large scale turntable dataset with detailed object pose and lighting information. We further show that the pretrained disCNN/AlexNet features on iLab- 20M generalize to object recognition on both Washington RGB-D and ImageNet datasets, and the pretrained disCNN features are significantly better than the pretrained AlexNet features for fine-tuning object recognition on the ImageNet dataset. |
| Tasks | Object Recognition |
| Published | 2016-07-20 |
| URL | http://arxiv.org/abs/1607.05851v3 |
| http://arxiv.org/pdf/1607.05851v3.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-recognize-objects-by-retaining |
| Repo | |
| Framework | |
Learning to Extract Motion from Videos in Convolutional Neural Networks
| Title | Learning to Extract Motion from Videos in Convolutional Neural Networks |
| Authors | Damien Teney, Martial Hebert |
| Abstract | This paper shows how to extract dense optical flow from videos with a convolutional neural network (CNN). The proposed model constitutes a potential building block for deeper architectures to allow using motion without resorting to an external algorithm, \eg for recognition in videos. We derive our network architecture from signal processing principles to provide desired invariances to image contrast, phase and texture. We constrain weights within the network to enforce strict rotation invariance and substantially reduce the number of parameters to learn. We demonstrate end-to-end training on only 8 sequences of the Middlebury dataset, orders of magnitude less than competing CNN-based motion estimation methods, and obtain comparable performance to classical methods on the Middlebury benchmark. Importantly, our method outputs a distributed representation of motion that allows representing multiple, transparent motions, and dynamic textures. Our contributions on network design and rotation invariance offer insights nonspecific to motion estimation. |
| Tasks | Motion Estimation, Optical Flow Estimation |
| Published | 2016-01-27 |
| URL | http://arxiv.org/abs/1601.07532v1 |
| http://arxiv.org/pdf/1601.07532v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-extract-motion-from-videos-in |
| Repo | |
| Framework | |
DeepProposals: Hunting Objects and Actions by Cascading Deep Convolutional Layers
| Title | DeepProposals: Hunting Objects and Actions by Cascading Deep Convolutional Layers |
| Authors | Amir Ghodrati, Ali Diba, Marco Pedersoli, Tinne Tuytelaars, Luc Van Gool |
| Abstract | In this paper, a new method for generating object and action proposals in images and videos is proposed. It builds on activations of different convolutional layers of a pretrained CNN, combining the localization accuracy of the early layers with the high informative-ness (and hence recall) of the later layers. To this end, we build an inverse cascade that, going backward from the later to the earlier convolutional layers of the CNN, selects the most promising locations and refines them in a coarse-to-fine manner. The method is efficient, because i) it re-uses the same features extracted for detection, ii) it aggregates features using integral images, and iii) it avoids a dense evaluation of the proposals thanks to the use of the inverse coarse-to-fine cascade. The method is also accurate. We show that our DeepProposals outperform most of the previously proposed object proposal and action proposal approaches and, when plugged into a CNN-based object detector, produce state-of-the-art detection performance. |
| Tasks | |
| Published | 2016-06-15 |
| URL | http://arxiv.org/abs/1606.04702v1 |
| http://arxiv.org/pdf/1606.04702v1.pdf | |
| PWC | https://paperswithcode.com/paper/deepproposals-hunting-objects-and-actions-by |
| Repo | |
| Framework | |
Optimal Rates for Multi-pass Stochastic Gradient Methods
| Title | Optimal Rates for Multi-pass Stochastic Gradient Methods |
| Authors | Junhong Lin, Lorenzo Rosasco |
| Abstract | We analyze the learning properties of the stochastic gradient method when multiple passes over the data and mini-batches are allowed. We study how regularization properties are controlled by the step-size, the number of passes and the mini-batch size. In particular, we consider the square loss and show that for a universal step-size choice, the number of passes acts as a regularization parameter, and optimal finite sample bounds can be achieved by early-stopping. Moreover, we show that larger step-sizes are allowed when considering mini-batches. Our analysis is based on a unifying approach, encompassing both batch and stochastic gradient methods as special cases. As a byproduct, we derive optimal convergence results for batch gradient methods (even in the non-attainable cases). |
| Tasks | |
| Published | 2016-05-28 |
| URL | http://arxiv.org/abs/1605.08882v3 |
| http://arxiv.org/pdf/1605.08882v3.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-rates-for-multi-pass-stochastic |
| Repo | |
| Framework | |
FLAG n’ FLARE: Fast Linearly-Coupled Adaptive Gradient Methods
| Title | FLAG n’ FLARE: Fast Linearly-Coupled Adaptive Gradient Methods |
| Authors | Xiang Cheng, Farbod Roosta-Khorasani, Stefan Palombo, Peter L. Bartlett, Michael W. Mahoney |
| Abstract | We consider first order gradient methods for effectively optimizing a composite objective in the form of a sum of smooth and, potentially, non-smooth functions. We present accelerated and adaptive gradient methods, called FLAG and FLARE, which can offer the best of both worlds. They can achieve the optimal convergence rate by attaining the optimal first-order oracle complexity for smooth convex optimization. Additionally, they can adaptively and non-uniformly re-scale the gradient direction to adapt to the limited curvature available and conform to the geometry of the domain. We show theoretically and empirically that, through the compounding effects of acceleration and adaptivity, FLAG and FLARE can be highly effective for many data fitting and machine learning applications. |
| Tasks | |
| Published | 2016-05-26 |
| URL | http://arxiv.org/abs/1605.08108v2 |
| http://arxiv.org/pdf/1605.08108v2.pdf | |
| PWC | https://paperswithcode.com/paper/flag-n-flare-fast-linearly-coupled-adaptive |
| Repo | |
| Framework | |
A Deep-Learning Approach for Operation of an Automated Realtime Flare Forecast
| Title | A Deep-Learning Approach for Operation of an Automated Realtime Flare Forecast |
| Authors | Yuko Hada-Muranushi, Takayuki Muranushi, Ayumi Asai, Daisuke Okanohara, Rudy Raymond, Gentaro Watanabe, Shigeru Nemoto, Kazunari Shibata |
| Abstract | Automated forecasts serve important role in space weather science, by providing statistical insights to flare-trigger mechanisms, and by enabling tailor-made forecasts and high-frequency forecasts. Only by realtime forecast we can experimentally measure the performance of flare-forecasting methods while confidently avoiding overlearning. We have been operating unmanned flare forecast service since August, 2015 that provides 24-hour-ahead forecast of solar flares, every 12 minutes. We report the method and prediction results of the system. |
| Tasks | |
| Published | 2016-06-06 |
| URL | http://arxiv.org/abs/1606.01587v1 |
| http://arxiv.org/pdf/1606.01587v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-deep-learning-approach-for-operation-of-an |
| Repo | |
| Framework | |
A Variational Analysis of Stochastic Gradient Algorithms
| Title | A Variational Analysis of Stochastic Gradient Algorithms |
| Authors | Stephan Mandt, Matthew D. Hoffman, David M. Blei |
| Abstract | Stochastic Gradient Descent (SGD) is an important algorithm in machine learning. With constant learning rates, it is a stochastic process that, after an initial phase of convergence, generates samples from a stationary distribution. We show that SGD with constant rates can be effectively used as an approximate posterior inference algorithm for probabilistic modeling. Specifically, we show how to adjust the tuning parameters of SGD such as to match the resulting stationary distribution to the posterior. This analysis rests on interpreting SGD as a continuous-time stochastic process and then minimizing the Kullback-Leibler divergence between its stationary distribution and the target posterior. (This is in the spirit of variational inference.) In more detail, we model SGD as a multivariate Ornstein-Uhlenbeck process and then use properties of this process to derive the optimal parameters. This theoretical framework also connects SGD to modern scalable inference algorithms; we analyze the recently proposed stochastic gradient Fisher scoring under this perspective. We demonstrate that SGD with properly chosen constant rates gives a new way to optimize hyperparameters in probabilistic models. |
| Tasks | |
| Published | 2016-02-08 |
| URL | http://arxiv.org/abs/1602.02666v1 |
| http://arxiv.org/pdf/1602.02666v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-variational-analysis-of-stochastic-gradient |
| Repo | |
| Framework | |
Data-Efficient Reinforcement Learning in Continuous-State POMDPs
| Title | Data-Efficient Reinforcement Learning in Continuous-State POMDPs |
| Authors | Rowan McAllister, Carl Edward Rasmussen |
| Abstract | We present a data-efficient reinforcement learning algorithm resistant to observation noise. Our method extends the highly data-efficient PILCO algorithm (Deisenroth & Rasmussen, 2011) into partially observed Markov decision processes (POMDPs) by considering the filtering process during policy evaluation. PILCO conducts policy search, evaluating each policy by first predicting an analytic distribution of possible system trajectories. We additionally predict trajectories w.r.t. a filtering process, achieving significantly higher performance than combining a filter with a policy optimised by the original (unfiltered) framework. Our test setup is the cartpole swing-up task with sensor noise, which involves nonlinear dynamics and requires nonlinear control. |
| Tasks | |
| Published | 2016-02-08 |
| URL | http://arxiv.org/abs/1602.02523v1 |
| http://arxiv.org/pdf/1602.02523v1.pdf | |
| PWC | https://paperswithcode.com/paper/data-efficient-reinforcement-learning-in-1 |
| Repo | |
| Framework | |
Fast and Simple Optimization for Poisson Likelihood Models
| Title | Fast and Simple Optimization for Poisson Likelihood Models |
| Authors | Niao He, Zaid Harchaoui, Yichen Wang, Le Song |
| Abstract | Poisson likelihood models have been prevalently used in imaging, social networks, and time series analysis. We propose fast, simple, theoretically-grounded, and versatile, optimization algorithms for Poisson likelihood modeling. The Poisson log-likelihood is concave but not Lipschitz-continuous. Since almost all gradient-based optimization algorithms rely on Lipschitz-continuity, optimizing Poisson likelihood models with a guarantee of convergence can be challenging, especially for large-scale problems. We present a new perspective allowing to efficiently optimize a wide range of penalized Poisson likelihood objectives. We show that an appropriate saddle point reformulation enjoys a favorable geometry and a smooth structure. Therefore, we can design a new gradient-based optimization algorithm with $O(1/t)$ convergence rate, in contrast to the usual $O(1/\sqrt{t})$ rate of non-smooth minimization alternatives. Furthermore, in order to tackle problems with large samples, we also develop a randomized block-decomposition variant that enjoys the same convergence rate yet more efficient iteration cost. Experimental results on several point process applications including social network estimation and temporal recommendation show that the proposed algorithm and its randomized block variant outperform existing methods both on synthetic and real-world datasets. |
| Tasks | Time Series, Time Series Analysis |
| Published | 2016-08-03 |
| URL | http://arxiv.org/abs/1608.01264v1 |
| http://arxiv.org/pdf/1608.01264v1.pdf | |
| PWC | https://paperswithcode.com/paper/fast-and-simple-optimization-for-poisson |
| Repo | |
| Framework | |
On Simulated Annealing Dedicated to Maximin Latin Hypercube Designs
| Title | On Simulated Annealing Dedicated to Maximin Latin Hypercube Designs |
| Authors | Pierre Bergé, Kaourintin Le Guiban, Arpad Rimmel, Joanna Tomasik |
| Abstract | The goal of our research was to enhance local search heuristics used to construct Latin Hypercube Designs. First, we introduce the \textit{1D-move} perturbation to improve the space exploration performed by these algorithms. Second, we propose a new evaluation function $\psi_{p,\sigma}$ specifically targeting the Maximin criterion. Exhaustive series of experiments with Simulated Annealing, which we used as a typically well-behaving local search heuristics, confirm that our goal was reached as the result we obtained surpasses the best scores reported in the literature. Furthermore, the $\psi_{p,\sigma}$ function seems very promising for a wide spectrum of optimization problems through the Maximin criterion. |
| Tasks | |
| Published | 2016-08-23 |
| URL | http://arxiv.org/abs/1608.07225v1 |
| http://arxiv.org/pdf/1608.07225v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-simulated-annealing-dedicated-to-maximin |
| Repo | |
| Framework | |
Phrase-based Machine Translation is State-of-the-Art for Automatic Grammatical Error Correction
| Title | Phrase-based Machine Translation is State-of-the-Art for Automatic Grammatical Error Correction |
| Authors | Marcin Junczys-Dowmunt, Roman Grundkiewicz |
| Abstract | In this work, we study parameter tuning towards the M^2 metric, the standard metric for automatic grammar error correction (GEC) tasks. After implementing M^2 as a scorer in the Moses tuning framework, we investigate interactions of dense and sparse features, different optimizers, and tuning strategies for the CoNLL-2014 shared task. We notice erratic behavior when optimizing sparse feature weights with M^2 and offer partial solutions. We find that a bare-bones phrase-based SMT setup with task-specific parameter-tuning outperforms all previously published results for the CoNLL-2014 test set by a large margin (46.37% M^2 over previously 41.75%, by an SMT system with neural features) while being trained on the same, publicly available data. Our newly introduced dense and sparse features widen that gap, and we improve the state-of-the-art to 49.49% M^2. |
| Tasks | Grammatical Error Correction, Machine Translation |
| Published | 2016-05-20 |
| URL | http://arxiv.org/abs/1605.06353v2 |
| http://arxiv.org/pdf/1605.06353v2.pdf | |
| PWC | https://paperswithcode.com/paper/phrase-based-machine-translation-is-state-of |
| Repo | |
| Framework | |
Fast Integral Image Estimation at 1% measurement rate
| Title | Fast Integral Image Estimation at 1% measurement rate |
| Authors | Kuldeep Kulkarni, Pavan Turaga |
| Abstract | We propose a framework called ReFInE to directly obtain integral image estimates from a very small number of spatially multiplexed measurements of the scene without iterative reconstruction of any auxiliary image, and demonstrate their practical utility in visual object tracking. Specifically, we design measurement matrices which are tailored to facilitate extremely fast estimation of the integral image, by using a single-shot linear operation on the measured vector. Leveraging a prior model for the images, we formulate a nuclear norm minimization problem with second order conic constraints to jointly obtain the measurement matrix and the linear operator. Through qualitative and quantitative experiments, we show that high quality integral image estimates can be obtained using our framework at very low measurement rates. Further, on a standard dataset of 50 videos, we present object tracking results which are comparable to the state-of-the-art methods, even at an extremely low measurement rate of 1%. |
| Tasks | Object Tracking, Visual Object Tracking |
| Published | 2016-01-27 |
| URL | http://arxiv.org/abs/1601.07258v1 |
| http://arxiv.org/pdf/1601.07258v1.pdf | |
| PWC | https://paperswithcode.com/paper/fast-integral-image-estimation-at-1 |
| Repo | |
| Framework | |
How Does the Low-Rank Matrix Decomposition Help Internal and External Learnings for Super-Resolution
| Title | How Does the Low-Rank Matrix Decomposition Help Internal and External Learnings for Super-Resolution |
| Authors | Shuang Wang, Bo Yue, Xuefeng Liang, Peiyuan Ji, Licheng Jiao |
| Abstract | Wisely utilizing the internal and external learning methods is a new challenge in super-resolution problem. To address this issue, we analyze the attributes of two methodologies and find two observations of their recovered details: 1) they are complementary in both feature space and image plane, 2) they distribute sparsely in the spatial space. These inspire us to propose a low-rank solution which effectively integrates two learning methods and then achieves a superior result. To fit this solution, the internal learning method and the external learning method are tailored to produce multiple preliminary results. Our theoretical analysis and experiment prove that the proposed low-rank solution does not require massive inputs to guarantee the performance, and thereby simplifying the design of two learning methods for the solution. Intensive experiments show the proposed solution improves the single learning method in both qualitative and quantitative assessments. Surprisingly, it shows more superior capability on noisy images and outperforms state-of-the-art methods. |
| Tasks | Super-Resolution |
| Published | 2016-04-06 |
| URL | http://arxiv.org/abs/1604.01497v3 |
| http://arxiv.org/pdf/1604.01497v3.pdf | |
| PWC | https://paperswithcode.com/paper/how-does-the-low-rank-matrix-decomposition |
| Repo | |
| Framework | |
Efficient functional ANOVA through wavelet-domain Markov groves
| Title | Efficient functional ANOVA through wavelet-domain Markov groves |
| Authors | Li Ma, Jacopo Soriano |
| Abstract | We introduce a wavelet-domain functional analysis of variance (fANOVA) method based on a Bayesian hierarchical model. The factor effects are modeled through a spike-and-slab mixture at each location-scale combination along with a normal-inverse-Gamma (NIG) conjugate setup for the coefficients and errors. A graphical model called the Markov grove (MG) is designed to jointly model the spike-and-slab statuses at all location-scale combinations, which incorporates the clustering of each factor effect in the wavelet-domain thereby allowing borrowing of strength across location and scale. The posterior of this NIG-MG model is analytically available through a pyramid algorithm of the same computational complexity as Mallat’s pyramid algorithm for discrete wavelet transform, i.e., linear in both the number of observations and the number of locations. Posterior probabilities of factor contributions can also be computed through pyramid recursion, and exact samples from the posterior can be drawn without MCMC. We investigate the performance of our method through extensive simulation and show that it outperforms existing wavelet-domain fANOVA methods in a variety of common settings. We apply the method to analyzing the orthosis data. |
| Tasks | |
| Published | 2016-02-12 |
| URL | http://arxiv.org/abs/1602.03990v2 |
| http://arxiv.org/pdf/1602.03990v2.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-functional-anova-through-wavelet |
| Repo | |
| Framework | |
Efficient and Consistent Robust Time Series Analysis
| Title | Efficient and Consistent Robust Time Series Analysis |
| Authors | Kush Bhatia, Prateek Jain, Parameswaran Kamalaruban, Purushottam Kar |
| Abstract | We study the problem of robust time series analysis under the standard auto-regressive (AR) time series model in the presence of arbitrary outliers. We devise an efficient hard thresholding based algorithm which can obtain a consistent estimate of the optimal AR model despite a large fraction of the time series points being corrupted. Our algorithm alternately estimates the corrupted set of points and the model parameters, and is inspired by recent advances in robust regression and hard-thresholding methods. However, a direct application of existing techniques is hindered by a critical difference in the time-series domain: each point is correlated with all previous points rendering existing tools inapplicable directly. We show how to overcome this hurdle using novel proof techniques. Using our techniques, we are also able to provide the first efficient and provably consistent estimator for the robust regression problem where a standard linear observation model with white additive noise is corrupted arbitrarily. We illustrate our methods on synthetic datasets and show that our methods indeed are able to consistently recover the optimal parameters despite a large fraction of points being corrupted. |
| Tasks | Time Series, Time Series Analysis |
| Published | 2016-07-01 |
| URL | http://arxiv.org/abs/1607.00146v1 |
| http://arxiv.org/pdf/1607.00146v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-and-consistent-robust-time-series |
| Repo | |
| Framework | |
