Paper Group ANR 504
The Power of Constraint Grammars Revisited. Approximate Steepest Coordinate Descent. Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps. Efficient K-Shot Learning with Regularized Deep Networks. MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation. In a Nutshell: Sequential Parameter Opt …
The Power of Constraint Grammars Revisited
| Title | The Power of Constraint Grammars Revisited |
| Authors | Anssi Yli-Jyrä |
| Abstract | Sequential Constraint Grammar (SCG) (Karlsson, 1990) and its extensions have lacked clear connections to formal language theory. The purpose of this article is to lay a foundation for these connections by simplifying the definition of strings processed by the grammar and by showing that Nonmonotonic SCG is undecidable and that derivations similar to the Generative Phonology exist. The current investigations propose resource bounds that restrict the generative power of SCG to a subset of context sensitive languages and present a strong finite-state condition for grammars as wholes. We show that a grammar is equivalent to a finite-state transducer if it is implemented with a Turing machine that runs in o(n log n) time. This condition opens new finite-state hypotheses and avenues for deeper analysis of SCG instances in the way inspired by Finite-State Phonology. |
| Tasks | |
| Published | 2017-07-17 |
| URL | http://arxiv.org/abs/1707.05115v1 |
| http://arxiv.org/pdf/1707.05115v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-power-of-constraint-grammars-revisited |
| Repo | |
| Framework | |
Approximate Steepest Coordinate Descent
| Title | Approximate Steepest Coordinate Descent |
| Authors | Sebastian U. Stich, Anant Raj, Martin Jaggi |
| Abstract | We propose a new selection rule for the coordinate selection in coordinate descent methods for huge-scale optimization. The efficiency of this novel scheme is provably better than the efficiency of uniformly random selection, and can reach the efficiency of steepest coordinate descent (SCD), enabling an acceleration of a factor of up to $n$, the number of coordinates. In many practical applications, our scheme can be implemented at no extra cost and computational efficiency very close to the faster uniform selection. Numerical experiments with Lasso and Ridge regression show promising improvements, in line with our theoretical guarantees. |
| Tasks | |
| Published | 2017-06-26 |
| URL | http://arxiv.org/abs/1706.08427v1 |
| http://arxiv.org/pdf/1706.08427v1.pdf | |
| PWC | https://paperswithcode.com/paper/approximate-steepest-coordinate-descent |
| Repo | |
| Framework | |
Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps
| Title | Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps |
| Authors | Liyuan Pan, Yuchao Dai, Miaomiao Liu, Fatih Porikli |
| Abstract | We aim at predicting a complete and high-resolution depth map from incomplete, sparse and noisy depth measurements. Existing methods handle this problem either by exploiting various regularizations on the depth maps directly or resorting to learning based methods. When the corresponding color images are available, the correlation between the depth maps and the color images are used to improve the completion performance, assuming the color images are clean and sharp. However, in real world dynamic scenes, color images are often blurry due to the camera motion and the moving objects in the scene. In this paper, we propose to tackle the problem of depth map completion by jointly exploiting the blurry color image sequences and the sparse depth map measurements, and present an energy minimization based formulation to simultaneously complete the depth maps, estimate the scene flow and deblur the color images. Our experimental evaluations on both outdoor and indoor scenarios demonstrate the state-of-the-art performance of our approach. |
| Tasks | |
| Published | 2017-11-27 |
| URL | http://arxiv.org/abs/1711.09501v1 |
| http://arxiv.org/pdf/1711.09501v1.pdf | |
| PWC | https://paperswithcode.com/paper/depth-map-completion-by-jointly-exploiting |
| Repo | |
| Framework | |
Efficient K-Shot Learning with Regularized Deep Networks
| Title | Efficient K-Shot Learning with Regularized Deep Networks |
| Authors | Donghyun Yoo, Haoqi Fan, Vishnu Naresh Boddeti, Kris M. Kitani |
| Abstract | Feature representations from pre-trained deep neural networks have been known to exhibit excellent generalization and utility across a variety of related tasks. Fine-tuning is by far the simplest and most widely used approach that seeks to exploit and adapt these feature representations to novel tasks with limited data. Despite the effectiveness of fine-tuning, itis often sub-optimal and requires very careful optimization to prevent severe over-fitting to small datasets. The problem of sub-optimality and over-fitting, is due in part to the large number of parameters used in a typical deep convolutional neural network. To address these problems, we propose a simple yet effective regularization method for fine-tuning pre-trained deep networks for the task of k-shot learning. To prevent overfitting, our key strategy is to cluster the model parameters while ensuring intra-cluster similarity and inter-cluster diversity of the parameters, effectively regularizing the dimensionality of the parameter search space. In particular, we identify groups of neurons within each layer of a deep network that shares similar activation patterns. When the network is to be fine-tuned for a classification task using only k examples, we propagate a single gradient to all of the neuron parameters that belong to the same group. The grouping of neurons is non-trivial as neuron activations depend on the distribution of the input data. To efficiently search for optimal groupings conditioned on the input data, we propose a reinforcement learning search strategy using recurrent networks to learn the optimal group assignments for each network layer. Experimental results show that our method can be easily applied to several popular convolutional neural networks and improve upon other state-of-the-art fine-tuning based k-shot learning strategies by more than10% |
| Tasks | |
| Published | 2017-10-06 |
| URL | http://arxiv.org/abs/1710.02277v1 |
| http://arxiv.org/pdf/1710.02277v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-k-shot-learning-with-regularized |
| Repo | |
| Framework | |
MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation
| Title | MirrorFlow: Exploiting Symmetries in Joint Optical Flow and Occlusion Estimation |
| Authors | Junhwa Hur, Stefan Roth |
| Abstract | Optical flow estimation is one of the most studied problems in computer vision, yet recent benchmark datasets continue to reveal problem areas of today’s approaches. Occlusions have remained one of the key challenges. In this paper, we propose a symmetric optical flow method to address the well-known chicken-and-egg relation between optical flow and occlusions. In contrast to many state-of-the-art methods that consider occlusions as outliers, possibly filtered out during post-processing, we highlight the importance of joint occlusion reasoning in the optimization and show how to utilize occlusion as an important cue for estimating optical flow. The key feature of our model is to fully exploit the symmetry properties that characterize optical flow and occlusions in the two consecutive images. Specifically through utilizing forward-backward consistency and occlusion-disocclusion symmetry in the energy, our model jointly estimates optical flow in both forward and backward direction, as well as consistent occlusion maps in both views. We demonstrate significant performance benefits on standard benchmarks, especially from the occlusion-disocclusion symmetry. On the challenging KITTI dataset we report the most accurate two-frame results to date. |
| Tasks | Optical Flow Estimation |
| Published | 2017-08-17 |
| URL | http://arxiv.org/abs/1708.05355v1 |
| http://arxiv.org/pdf/1708.05355v1.pdf | |
| PWC | https://paperswithcode.com/paper/mirrorflow-exploiting-symmetries-in-joint |
| Repo | |
| Framework | |
In a Nutshell: Sequential Parameter Optimization
| Title | In a Nutshell: Sequential Parameter Optimization |
| Authors | Thomas Bartz-Beielstein, Lorenzo Gentile, Martin Zaefferer |
| Abstract | The performance of optimization algorithms relies crucially on their parameterizations. Finding good parameter settings is called algorithm tuning. Using a simple simulated annealing algorithm, we will demonstrate how optimization algorithms can be tuned using the sequential parameter optimization toolbox (SPOT). SPOT provides several tools for automated and interactive tuning. The underling concepts of the SPOT approach are explained. This includes key techniques such as exploratory fitness landscape analysis and response surface methodology. Many examples illustrate how SPOT can be used for understanding the performance of algorithms and gaining insight into algorithm’s behavior. Furthermore, we demonstrate how SPOT can be used as an optimizer and how a sophisticated ensemble approach is able to combine several meta models via stacking. |
| Tasks | |
| Published | 2017-12-12 |
| URL | http://arxiv.org/abs/1712.04076v1 |
| http://arxiv.org/pdf/1712.04076v1.pdf | |
| PWC | https://paperswithcode.com/paper/in-a-nutshell-sequential-parameter |
| Repo | |
| Framework | |
Experimental Phase Estimation Enhanced By Machine Learning
| Title | Experimental Phase Estimation Enhanced By Machine Learning |
| Authors | Alessandro Lumino, Emanuele Polino, Adil S. Rab, Giorgio Milani, Nicolò Spagnolo, Nathan Wiebe, Fabio Sciarrino |
| Abstract | Phase estimation protocols provide a fundamental benchmark for the field of quantum metrology. The latter represents one of the most relevant applications of quantum theory, potentially enabling the capability of measuring unknown physical parameters with improved precision over classical strategies. Within this context, most theoretical and experimental studies have focused on determining the fundamental bounds and how to achieve them in the asymptotic regime where a large number of resources is employed. However, in most applications it is necessary to achieve optimal precisions by performing only a limited number of measurements. To this end, machine learning techniques can be applied as a powerful optimization tool. Here, we implement experimentally single-photon adaptive phase estimation protocols enhanced by machine learning, showing the capability of reaching optimal precision after a small number of trials. In particular, we introduce a new approach for Bayesian estimation that exhibit best performances for very low number of photons N. Furthermore, we study the resilience to noise of the tested methods, showing that the optimized Bayesian approach is very robust in the presence of imperfections. Application of this methodology can be envisaged in the more general multiparameter case, that represents a paradigmatic scenario for several tasks including imaging or Hamiltonian learning. |
| Tasks | |
| Published | 2017-12-20 |
| URL | http://arxiv.org/abs/1712.07570v1 |
| http://arxiv.org/pdf/1712.07570v1.pdf | |
| PWC | https://paperswithcode.com/paper/experimental-phase-estimation-enhanced-by |
| Repo | |
| Framework | |
Exponential scaling of neural algorithms - a future beyond Moore’s Law?
| Title | Exponential scaling of neural algorithms - a future beyond Moore’s Law? |
| Authors | James B. Aimone |
| Abstract | Although the brain has long been considered a potential inspiration for future computing, Moore’s Law - the scaling property that has seen revolutions in technologies ranging from supercomputers to smart phones - has largely been driven by advances in materials science. As the ability to miniaturize transistors is coming to an end, there is increasing attention on new approaches to computation, including renewed enthusiasm around the potential of neural computation. This paper describes how recent advances in neurotechnologies, many of which have been aided by computing’s rapid progression over recent decades, are now reigniting this opportunity to bring neural computation insights into broader computing applications. As we understand more about the brain, our ability to motivate new computing paradigms with continue to progress. These new approaches to computing, which we are already seeing in techniques such as deep learning and neuromorphic hardware, will themselves improve our ability to learn about the brain and accordingly can be projected to give rise to even further insights. This paper will describe how this positive feedback has the potential to change the complexion of how computing sciences and neurosciences interact, and suggests that the next form of exponential scaling in computing may emerge from our progressive understanding of the brain. |
| Tasks | |
| Published | 2017-05-04 |
| URL | http://arxiv.org/abs/1705.02042v2 |
| http://arxiv.org/pdf/1705.02042v2.pdf | |
| PWC | https://paperswithcode.com/paper/exponential-scaling-of-neural-algorithms-a |
| Repo | |
| Framework | |
Two-Dimensional Indirect Binary Search for the Positive One-in-Three Satisfiability Problem
| Title | Two-Dimensional Indirect Binary Search for the Positive One-in-Three Satisfiability Problem |
| Authors | Shunichi Matsubara |
| Abstract | In this paper, we propose an algorithm for the positive one-in-three satisfiability problem (Pos1in3SAT). The proposed algorithm can efficiently decide the existence of a satisfying assignment in all assignments for a given formula by using a 2-dimensional binary search method without constructing an exponential number of assignments. |
| Tasks | |
| Published | 2017-08-28 |
| URL | http://arxiv.org/abs/1708.08377v2 |
| http://arxiv.org/pdf/1708.08377v2.pdf | |
| PWC | https://paperswithcode.com/paper/two-dimensional-indirect-binary-search-for |
| Repo | |
| Framework | |
Linguistic Matrix Theory
| Title | Linguistic Matrix Theory |
| Authors | Dimitrios Kartsaklis, Sanjaye Ramgoolam, Mehrnoosh Sadrzadeh |
| Abstract | Recent research in computational linguistics has developed algorithms which associate matrices with adjectives and verbs, based on the distribution of words in a corpus of text. These matrices are linear operators on a vector space of context words. They are used to construct the meaning of composite expressions from that of the elementary constituents, forming part of a compositional distributional approach to semantics. We propose a Matrix Theory approach to this data, based on permutation symmetry along with Gaussian weights and their perturbations. A simple Gaussian model is tested against word matrices created from a large corpus of text. We characterize the cubic and quartic departures from the model, which we propose, alongside the Gaussian parameters, as signatures for comparison of linguistic corpora. We propose that perturbed Gaussian models with permutation symmetry provide a promising framework for characterizing the nature of universality in the statistical properties of word matrices. The matrix theory framework developed here exploits the view of statistics as zero dimensional perturbative quantum field theory. It perceives language as a physical system realizing a universality class of matrix statistics characterized by permutation symmetry. |
| Tasks | |
| Published | 2017-03-28 |
| URL | http://arxiv.org/abs/1703.10252v1 |
| http://arxiv.org/pdf/1703.10252v1.pdf | |
| PWC | https://paperswithcode.com/paper/linguistic-matrix-theory |
| Repo | |
| Framework | |
Shape Parameter Estimation
| Title | Shape Parameter Estimation |
| Authors | Peng Zheng, Aleksandr Y. Aravkin, Karthikeyan Natesan Ramamurthy |
| Abstract | Performance of machine learning approaches depends strongly on the choice of misfit penalty, and correct choice of penalty parameters, such as the threshold of the Huber function. These parameters are typically chosen using expert knowledge, cross-validation, or black-box optimization, which are time consuming for large-scale applications. We present a principled, data-driven approach to simultaneously learn the model pa- rameters and the misfit penalty parameters. We discuss theoretical properties of these joint inference problems, and develop algorithms for their solution. We show synthetic examples of automatic parameter tuning for piecewise linear-quadratic (PLQ) penalties, and use the approach to develop a self-tuning robust PCA formulation for background separation. |
| Tasks | |
| Published | 2017-06-06 |
| URL | http://arxiv.org/abs/1706.01865v1 |
| http://arxiv.org/pdf/1706.01865v1.pdf | |
| PWC | https://paperswithcode.com/paper/shape-parameter-estimation |
| Repo | |
| Framework | |
Design and Processing of Invertible Orientation Scores of 3D Images for Enhancement of Complex Vasculature
| Title | Design and Processing of Invertible Orientation Scores of 3D Images for Enhancement of Complex Vasculature |
| Authors | M. H. J. Janssen, A. J. E. M. Janssen, E. J. Bekkers, J. Olivan Bescos, R. Duits |
| Abstract | The enhancement and detection of elongated structures in noisy image data is relevant for many biomedical imaging applications. To handle complex crossing structures in 2D images, 2D orientation scores $U: \mathbb{R} ^ 2\times S ^ 1 \rightarrow \mathbb{C}$ were introduced, which already showed their use in a variety of applications. Here we extend this work to 3D orientation scores $U: \mathbb{R} ^ 3 \times S ^ 2\rightarrow \mathbb{C}$. First, we construct the orientation score from a given dataset, which is achieved by an invertible coherent state type of transform. For this transformation we introduce 3D versions of the 2D cake-wavelets, which are complex wavelets that can simultaneously detect oriented structures and oriented edges. Here we introduce two types of cake-wavelets, the first uses a discrete Fourier transform, the second is designed in the 3D generalized Zernike basis, allowing us to calculate analytical expressions for the spatial filters. Finally, we show two applications of the orientation score transformation. In the first application we propose an extension of crossing-preserving coherence enhancing diffusion via our invertible orientation scores of 3D images which we apply to real medical image data. In the second one we develop a new tubularity measure using 3D orientation scores and apply the tubularity measure to both artificial and real medical data. |
| Tasks | |
| Published | 2017-07-07 |
| URL | http://arxiv.org/abs/1707.02191v3 |
| http://arxiv.org/pdf/1707.02191v3.pdf | |
| PWC | https://paperswithcode.com/paper/design-and-processing-of-invertible |
| Repo | |
| Framework | |
Probabilistic Combination of Noisy Points and Planes for RGB-D Odometry
| Title | Probabilistic Combination of Noisy Points and Planes for RGB-D Odometry |
| Authors | Pedro F. Proença, Yang Gao |
| Abstract | This work proposes a visual odometry method that combines points and plane primitives, extracted from a noisy depth camera. Depth measurement uncertainty is modelled and propagated through the extraction of geometric primitives to the frame-to-frame motion estimation, where pose is optimized by weighting the residuals of 3D point and planes matches, according to their uncertainties. Results on an RGB-D dataset show that the combination of points and planes, through the proposed method, is able to perform well in poorly textured environments, where point-based odometry is bound to fail. |
| Tasks | Motion Estimation, Visual Odometry |
| Published | 2017-05-18 |
| URL | http://arxiv.org/abs/1705.06516v1 |
| http://arxiv.org/pdf/1705.06516v1.pdf | |
| PWC | https://paperswithcode.com/paper/probabilistic-combination-of-noisy-points-and |
| Repo | |
| Framework | |
Hyperplane Clustering Via Dual Principal Component Pursuit
| Title | Hyperplane Clustering Via Dual Principal Component Pursuit |
| Authors | Manolis C. Tsakiris, Rene Vidal |
| Abstract | We extend the theoretical analysis of a recently proposed single subspace learning algorithm, called Dual Principal Component Pursuit (DPCP), to the case where the data are drawn from of a union of hyperplanes. To gain insight into the properties of the $\ell_1$ non-convex problem associated with DPCP, we develop a geometric analysis of a closely related continuous optimization problem. Then transferring this analysis to the discrete problem, our results state that as long as the hyperplanes are sufficiently separated, the dominant hyperplane is sufficiently dominant and the points are uniformly distributed inside the associated hyperplanes, then the non-convex DPCP problem has a unique global solution, equal to the normal vector of the dominant hyperplane. This suggests the correctness of a sequential hyperplane learning algorithm based on DPCP. A thorough experimental evaluation reveals that hyperplane learning schemes based on DPCP dramatically improve over the state-of-the-art methods for the case of synthetic data, while are competitive to the state-of-the-art in the case of 3D plane clustering for Kinect data. |
| Tasks | |
| Published | 2017-06-06 |
| URL | http://arxiv.org/abs/1706.01604v2 |
| http://arxiv.org/pdf/1706.01604v2.pdf | |
| PWC | https://paperswithcode.com/paper/hyperplane-clustering-via-dual-principal |
| Repo | |
| Framework | |
A Data and Model-Parallel, Distributed and Scalable Framework for Training of Deep Networks in Apache Spark
| Title | A Data and Model-Parallel, Distributed and Scalable Framework for Training of Deep Networks in Apache Spark |
| Authors | Disha Shrivastava, Santanu Chaudhury, Dr. Jayadeva |
| Abstract | Training deep networks is expensive and time-consuming with the training period increasing with data size and growth in model parameters. In this paper, we provide a framework for distributed training of deep networks over a cluster of CPUs in Apache Spark. The framework implements both Data Parallelism and Model Parallelism making it suitable to use for deep networks which require huge training data and model parameters which are too big to fit into the memory of a single machine. It can be scaled easily over a cluster of cheap commodity hardware to attain significant speedup and obtain better results making it quite economical as compared to farm of GPUs and supercomputers. We have proposed a new algorithm for training of deep networks for the case when the network is partitioned across the machines (Model Parallelism) along with detailed cost analysis and proof of convergence of the same. We have developed implementations for Fully-Connected Feedforward Networks, Convolutional Neural Networks, Recurrent Neural Networks and Long Short-Term Memory architectures. We present the results of extensive simulations demonstrating the speedup and accuracy obtained by our framework for different sizes of the data and model parameters with variation in the number of worker cores/partitions; thereby showing that our proposed framework can achieve significant speedup (upto 11X for CNN) and is also quite scalable. |
| Tasks | |
| Published | 2017-08-19 |
| URL | http://arxiv.org/abs/1708.05840v1 |
| http://arxiv.org/pdf/1708.05840v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-data-and-model-parallel-distributed-and |
| Repo | |
| Framework | |
