Paper Group ANR 463
How useful is photo-realistic rendering for visual learning?. Double Coupled Canonical Polyadic Decomposition for Joint Blind Source Separation. Functorial Hierarchical Clustering with Overlaps. Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks. Surpassing Gradient Descent Provably: A Cyclic Incremental Method wit …
How useful is photo-realistic rendering for visual learning?
| Title | How useful is photo-realistic rendering for visual learning? |
| Authors | Yair Movshovitz-Attias, Takeo Kanade, Yaser Sheikh |
| Abstract | Data seems cheap to get, and in many ways it is, but the process of creating a high quality labeled dataset from a mass of data is time-consuming and expensive. With the advent of rich 3D repositories, photo-realistic rendering systems offer the opportunity to provide nearly limitless data. Yet, their primary value for visual learning may be the quality of the data they can provide rather than the quantity. Rendering engines offer the promise of perfect labels in addition to the data: what the precise camera pose is; what the precise lighting location, temperature, and distribution is; what the geometry of the object is. In this work we focus on semi-automating dataset creation through use of synthetic data and apply this method to an important task – object viewpoint estimation. Using state-of-the-art rendering software we generate a large labeled dataset of cars rendered densely in viewpoint space. We investigate the effect of rendering parameters on estimation performance and show realism is important. We show that generalizing from synthetic data is not harder than the domain adaptation required between two real-image datasets and that combining synthetic images with a small amount of real data improves estimation accuracy. |
| Tasks | Domain Adaptation, Viewpoint Estimation |
| Published | 2016-03-26 |
| URL | http://arxiv.org/abs/1603.08152v2 |
| http://arxiv.org/pdf/1603.08152v2.pdf | |
| PWC | https://paperswithcode.com/paper/how-useful-is-photo-realistic-rendering-for |
| Repo | |
| Framework | |
Double Coupled Canonical Polyadic Decomposition for Joint Blind Source Separation
| Title | Double Coupled Canonical Polyadic Decomposition for Joint Blind Source Separation |
| Authors | Xiao-Feng Gong, Qiu-Hua Lin, Feng-Yu Cong, Lieven De Lathauwer |
| Abstract | Joint blind source separation (J-BSS) is an emerging data-driven technique for multi-set data-fusion. In this paper, J-BSS is addressed from a tensorial perspective. We show how, by using second-order multi-set statistics in J-BSS, a specific double coupled canonical polyadic decomposition (DC-CPD) problem can be formulated. We propose an algebraic DC-CPD algorithm based on a coupled rank-1 detection mapping. This algorithm converts a possibly underdetermined DC-CPD to a set of overdetermined CPDs. The latter can be solved algebraically via a generalized eigenvalue decomposition based scheme. Therefore, this algorithm is deterministic and returns the exact solution in the noiseless case. In the noisy case, it can be used to effectively initialize optimization based DC-CPD algorithms. In addition, we obtain the determini- stic and generic uniqueness conditions for DC-CPD, which are shown to be more relaxed than their CPD counterpart. Experiment results are given to illustrate the superiority of DC-CPD over standard CPD based BSS methods and several existing J-BSS methods, with regards to uniqueness and accuracy. |
| Tasks | |
| Published | 2016-12-30 |
| URL | http://arxiv.org/abs/1612.09466v4 |
| http://arxiv.org/pdf/1612.09466v4.pdf | |
| PWC | https://paperswithcode.com/paper/double-coupled-canonical-polyadic |
| Repo | |
| Framework | |
Functorial Hierarchical Clustering with Overlaps
| Title | Functorial Hierarchical Clustering with Overlaps |
| Authors | Jared Culbertson, Dan P. Guralnik, Peter F. Stiller |
| Abstract | This work draws inspiration from three important sources of research on dissimilarity-based clustering and intertwines those three threads into a consistent principled functorial theory of clustering. Those three are the overlapping clustering of Jardine and Sibson, the functorial approach of Carlsson and M'{e}moli to partition-based clustering, and the Isbell/Dress school’s study of injective envelopes. Carlsson and M'{e}moli introduce the idea of viewing clustering methods as functors from a category of metric spaces to a category of clusters, with functoriality subsuming many desirable properties. Our first series of results extends their theory of functorial clustering schemes to methods that allow overlapping clusters in the spirit of Jardine and Sibson. This obviates some of the unpleasant effects of chaining that occur, for example with single-linkage clustering. We prove an equivalence between these general overlapping clustering functors and projections of weight spaces to what we term clustering domains, by focusing on the order structure determined by the morphisms. As a specific application of this machinery, we are able to prove that there are no functorial projections to cut metrics, or even to tree metrics. Finally, although we focus less on the construction of clustering methods (clustering domains) derived from injective envelopes, we lay out some preliminary results, that hopefully will give a feel for how the third leg of the stool comes into play. |
| Tasks | |
| Published | 2016-09-08 |
| URL | http://arxiv.org/abs/1609.02513v2 |
| http://arxiv.org/pdf/1609.02513v2.pdf | |
| PWC | https://paperswithcode.com/paper/functorial-hierarchical-clustering-with |
| Repo | |
| Framework | |
Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks
| Title | Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks |
| Authors | Farkhondeh Kiaee, Christian Gagné, Mahdieh Abbasi |
| Abstract | The storage and computation requirements of Convolutional Neural Networks (CNNs) can be prohibitive for exploiting these models over low-power or embedded devices. This paper reduces the computational complexity of the CNNs by minimizing an objective function, including the recognition loss that is augmented with a sparsity-promoting penalty term. The sparsity structure of the network is identified using the Alternating Direction Method of Multipliers (ADMM), which is widely used in large optimization problems. This method alternates between promoting the sparsity of the network and optimizing the recognition performance, which allows us to exploit the two-part structure of the corresponding objective functions. In particular, we take advantage of the separability of the sparsity-inducing penalty functions to decompose the minimization problem into sub-problems that can be solved sequentially. Applying our method to a variety of state-of-the-art CNN models, our proposed method is able to simplify the original model, generating models with less computation and fewer parameters, while maintaining and often improving generalization performance. Accomplishments on a variety of models strongly verify that our proposed ADMM-based method can be a very useful tool for simplifying and improving deep CNNs. |
| Tasks | |
| Published | 2016-11-05 |
| URL | http://arxiv.org/abs/1611.01590v3 |
| http://arxiv.org/pdf/1611.01590v3.pdf | |
| PWC | https://paperswithcode.com/paper/alternating-direction-method-of-multipliers-1 |
| Repo | |
| Framework | |
Surpassing Gradient Descent Provably: A Cyclic Incremental Method with Linear Convergence Rate
| Title | Surpassing Gradient Descent Provably: A Cyclic Incremental Method with Linear Convergence Rate |
| Authors | Aryan Mokhtari, Mert Gürbüzbalaban, Alejandro Ribeiro |
| Abstract | Recently, there has been growing interest in developing optimization methods for solving large-scale machine learning problems. Most of these problems boil down to the problem of minimizing an average of a finite set of smooth and strongly convex functions where the number of functions $n$ is large. Gradient descent method (GD) is successful in minimizing convex problems at a fast linear rate; however, it is not applicable to the considered large-scale optimization setting because of the high computational complexity. Incremental methods resolve this drawback of gradient methods by replacing the required gradient for the descent direction with an incremental gradient approximation. They operate by evaluating one gradient per iteration and executing the average of the $n$ available gradients as a gradient approximate. Although, incremental methods reduce the computational cost of GD, their convergence rates do not justify their advantage relative to GD in terms of the total number of gradient evaluations until convergence. In this paper, we introduce a Double Incremental Aggregated Gradient method (DIAG) that computes the gradient of only one function at each iteration, which is chosen based on a cyclic scheme, and uses the aggregated average gradient of all the functions to approximate the full gradient. The iterates of the proposed DIAG method uses averages of both iterates and gradients in oppose to classic incremental methods that utilize gradient averages but do not utilize iterate averages. We prove that not only the proposed DIAG method converges linearly to the optimal solution, but also its linear convergence factor justifies the advantage of incremental methods on GD. In particular, we prove that the worst case performance of DIAG is better than the worst case performance of GD. |
| Tasks | |
| Published | 2016-11-01 |
| URL | http://arxiv.org/abs/1611.00347v2 |
| http://arxiv.org/pdf/1611.00347v2.pdf | |
| PWC | https://paperswithcode.com/paper/surpassing-gradient-descent-provably-a-cyclic |
| Repo | |
| Framework | |
Neural Multi-Source Morphological Reinflection
| Title | Neural Multi-Source Morphological Reinflection |
| Authors | Katharina Kann, Ryan Cotterell, Hinrich Schütze |
| Abstract | We explore the task of multi-source morphological reinflection, which generalizes the standard, single-source version. The input consists of (i) a target tag and (ii) multiple pairs of source form and source tag for a lemma. The motivation is that it is beneficial to have access to more than one source form since different source forms can provide complementary information, e.g., different stems. We further present a novel extension to the encoder- decoder recurrent neural architecture, consisting of multiple encoders, to better solve the task. We show that our new architecture outperforms single-source reinflection models and publish our dataset for multi-source morphological reinflection to facilitate future research. |
| Tasks | |
| Published | 2016-12-19 |
| URL | http://arxiv.org/abs/1612.06027v3 |
| http://arxiv.org/pdf/1612.06027v3.pdf | |
| PWC | https://paperswithcode.com/paper/neural-multi-source-morphological |
| Repo | |
| Framework | |
Self-adaptation of Mutation Rates in Non-elitist Populations
| Title | Self-adaptation of Mutation Rates in Non-elitist Populations |
| Authors | Duc-Cuong Dang, Per Kristian Lehre |
| Abstract | The runtime of evolutionary algorithms (EAs) depends critically on their parameter settings, which are often problem-specific. Automated schemes for parameter tuning have been developed to alleviate the high costs of manual parameter tuning. Experimental results indicate that self-adaptation, where parameter settings are encoded in the genomes of individuals, can be effective in continuous optimisation. However, results in discrete optimisation have been less conclusive. Furthermore, a rigorous runtime analysis that explains how self-adaptation can lead to asymptotic speedups has been missing. This paper provides the first such analysis for discrete, population-based EAs. We apply level-based analysis to show how a self-adaptive EA is capable of fine-tuning its mutation rate, leading to exponential speedups over EAs using fixed mutation rates. |
| Tasks | |
| Published | 2016-06-17 |
| URL | http://arxiv.org/abs/1606.05551v1 |
| http://arxiv.org/pdf/1606.05551v1.pdf | |
| PWC | https://paperswithcode.com/paper/self-adaptation-of-mutation-rates-in-non |
| Repo | |
| Framework | |
Recovering the number of clusters in data sets with noise features using feature rescaling factors
| Title | Recovering the number of clusters in data sets with noise features using feature rescaling factors |
| Authors | Renato Cordeiro de Amorim, Christian Hennig |
| Abstract | In this paper we introduce three methods for re-scaling data sets aiming at improving the likelihood of clustering validity indexes to return the true number of spherical Gaussian clusters with additional noise features. Our method obtains feature re-scaling factors taking into account the structure of a given data set and the intuitive idea that different features may have different degrees of relevance at different clusters. We experiment with the Silhouette (using squared Euclidean, Manhattan, and the p$^{th}$ power of the Minkowski distance), Dunn’s, Calinski-Harabasz and Hartigan indexes on data sets with spherical Gaussian clusters with and without noise features. We conclude that our methods indeed increase the chances of estimating the true number of clusters in a data set. |
| Tasks | |
| Published | 2016-02-22 |
| URL | http://arxiv.org/abs/1602.06989v1 |
| http://arxiv.org/pdf/1602.06989v1.pdf | |
| PWC | https://paperswithcode.com/paper/recovering-the-number-of-clusters-in-data |
| Repo | |
| Framework | |
HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
| Title | HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment |
| Authors | Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, Anna Korhonen |
| Abstract | We introduce HyperLex - a dataset and evaluation resource that quantifies the extent of of the semantic category membership, that is, type-of relation also known as hyponymy-hypernymy or lexical entailment (LE) relation between 2,616 concept pairs. Cognitive psychology research has established that typicality and category/class membership are computed in human semantic memory as a gradual rather than binary relation. Nevertheless, most NLP research, and existing large-scale invetories of concept category membership (WordNet, DBPedia, etc.) treat category membership and LE as binary. To address this, we asked hundreds of native English speakers to indicate typicality and strength of category membership between a diverse range of concept pairs on a crowdsourcing platform. Our results confirm that category membership and LE are indeed more gradual than binary. We then compare these human judgements with the predictions of automatic systems, which reveals a huge gap between human performance and state-of-the-art LE, distributional and representation learning models, and substantial differences between the models themselves. We discuss a pathway for improving semantic models to overcome this discrepancy, and indicate future application areas for improved graded LE systems. |
| Tasks | Representation Learning |
| Published | 2016-08-06 |
| URL | http://arxiv.org/abs/1608.02117v2 |
| http://arxiv.org/pdf/1608.02117v2.pdf | |
| PWC | https://paperswithcode.com/paper/hyperlex-a-large-scale-evaluation-of-graded |
| Repo | |
| Framework | |
Superconducting optoelectronic circuits for neuromorphic computing
| Title | Superconducting optoelectronic circuits for neuromorphic computing |
| Authors | Jeffrey M. Shainline, Sonia M. Buckley, Richard P. Mirin, Sae Woo Nam |
| Abstract | Neural networks have proven effective for solving many difficult computational problems. Implementing complex neural networks in software is very computationally expensive. To explore the limits of information processing, it will be necessary to implement new hardware platforms with large numbers of neurons, each with a large number of connections to other neurons. Here we propose a hybrid semiconductor-superconductor hardware platform for the implementation of neural networks and large-scale neuromorphic computing. The platform combines semiconducting few-photon light-emitting diodes with superconducting-nanowire single-photon detectors to behave as spiking neurons. These processing units are connected via a network of optical waveguides, and variable weights of connection can be implemented using several approaches. The use of light as a signaling mechanism overcomes fanout and parasitic constraints on electrical signals while simultaneously introducing physical degrees of freedom which can be employed for computation. The use of supercurrents achieves the low power density necessary to scale to systems with enormous entropy. The proposed processing units can operate at speeds of at least $20$ MHz with fully asynchronous activity, light-speed-limited latency, and power densities on the order of 1 mW/cm$^2$ for neurons with 700 connections operating at full speed at 2 K. The processing units achieve an energy efficiency of $\approx 20$ aJ per synapse event. By leveraging multilayer photonics with deposited waveguides and superconductors with feature sizes $>$ 100 nm, this approach could scale to systems with massive interconnectivity and complexity for advanced computing as well as explorations of information processing capacity in systems with an enormous number of information-bearing microstates. |
| Tasks | |
| Published | 2016-09-30 |
| URL | http://arxiv.org/abs/1610.00053v2 |
| http://arxiv.org/pdf/1610.00053v2.pdf | |
| PWC | https://paperswithcode.com/paper/superconducting-optoelectronic-circuits-for |
| Repo | |
| Framework | |
A New Parallel Algorithm for Two-Pass Connected Component Labeling
| Title | A New Parallel Algorithm for Two-Pass Connected Component Labeling |
| Authors | Siddharth Gupta, Diana Palsetia, Md. Mostofa Ali Patwary, Ankit Agrawal, Alok Choudhary |
| Abstract | Connected Component Labeling (CCL) is an important step in pattern recognition and image processing. It assigns labels to the pixels such that adjacent pixels sharing the same features are assigned the same label. Typically, CCL requires several passes over the data. We focus on two-pass technique where each pixel is given a provisional label in the first pass whereas an actual label is assigned in the second pass. We present a scalable parallel two-pass CCL algorithm, called PAREMSP, which employs a scan strategy and the best union-find technique called REMSP, which uses REM’s algorithm for storing label equivalence information of pixels in a 2-D image. In the first pass, we divide the image among threads and each thread runs the scan phase along with REMSP simultaneously. In the second phase, we assign the final labels to the pixels. As REMSP is easily parallelizable, we use the parallel version of REMSP for merging the pixels on the boundary. Our experiments show the scalability of PAREMSP achieving speedups up to $20.1$ using $24$ cores on shared memory architecture using OpenMP for an image of size $465.20$ MB. We find that our proposed parallel algorithm achieves linear scaling for a large resolution fixed problem size as the number of processing elements are increased. Additionally, the parallel algorithm does not make use of any hardware specific routines, and thus is highly portable. |
| Tasks | |
| Published | 2016-06-20 |
| URL | http://arxiv.org/abs/1606.05973v1 |
| http://arxiv.org/pdf/1606.05973v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-new-parallel-algorithm-for-two-pass |
| Repo | |
| Framework | |
Learning Sentence Representation with Guidance of Human Attention
| Title | Learning Sentence Representation with Guidance of Human Attention |
| Authors | Shaonan Wang, Jiajun Zhang, Chengqing Zong |
| Abstract | Recently, much progress has been made in learning general-purpose sentence representations that can be used across domains. However, most of the existing models typically treat each word in a sentence equally. In contrast, extensive studies have proven that human read sentences efficiently by making a sequence of fixation and saccades. This motivates us to improve sentence representations by assigning different weights to the vectors of the component words, which can be treated as an attention mechanism on single sentences. To that end, we propose two novel attention models, in which the attention weights are derived using significant predictors of human reading time, i.e., Surprisal, POS tags and CCG supertags. The extensive experiments demonstrate that the proposed methods significantly improve upon the state-of-the-art sentence representation models. |
| Tasks | |
| Published | 2016-09-29 |
| URL | http://arxiv.org/abs/1609.09189v2 |
| http://arxiv.org/pdf/1609.09189v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-sentence-representation-with |
| Repo | |
| Framework | |
Adaptable Precomputation for Random Walker Image Segmentation and Registration
| Title | Adaptable Precomputation for Random Walker Image Segmentation and Registration |
| Authors | Shawn Andrews, Ghassan Hamarneh |
| Abstract | The random walker (RW) algorithm is used for both image segmentation and registration, and possesses several useful properties that make it popular in medical imaging, such as being globally optimizable, allowing user interaction, and providing uncertainty information. The RW algorithm defines a weighted graph over an image and uses the graph’s Laplacian matrix to regularize its solutions. This regularization reduces to solving a large system of equations, which may be excessively time consuming in some applications, such as when interacting with a human user. Techniques have been developed that precompute eigenvectors of a Laplacian offline, after image acquisition but before any analysis, in order speed up the RW algorithm online, when segmentation or registration is being performed. However, precomputation requires certain algorithm parameters be fixed offline, limiting their flexibility. In this paper, we develop techniques to update the precomputed data online when RW parameters are altered. Specifically, we dynamically determine the number of eigenvectors needed for a desired accuracy based on user input, and derive update equations for the eigenvectors when the edge weights or topology of the image graph are changed. We present results demonstrating that our techniques make RW with precomputation much more robust to offline settings while only sacrificing minimal accuracy. |
| Tasks | Semantic Segmentation |
| Published | 2016-07-14 |
| URL | http://arxiv.org/abs/1607.04174v1 |
| http://arxiv.org/pdf/1607.04174v1.pdf | |
| PWC | https://paperswithcode.com/paper/adaptable-precomputation-for-random-walker |
| Repo | |
| Framework | |
What Is Around The Camera?
| Title | What Is Around The Camera? |
| Authors | Stamatios Georgoulis, Konstantinos Rematas, Tobias Ritschel, Mario Fritz, Tinne Tuytelaars, Luc Van Gool |
| Abstract | How much does a single image reveal about the environment it was taken in? In this paper, we investigate how much of that information can be retrieved from a foreground object, combined with the background (i.e. the visible part of the environment). Assuming it is not perfectly diffuse, the foreground object acts as a complexly shaped and far-from-perfect mirror. An additional challenge is that its appearance confounds the light coming from the environment with the unknown materials it is made of. We propose a learning-based approach to predict the environment from multiple reflectance maps that are computed from approximate surface normals. The proposed method allows us to jointly model the statistics of environments and material properties. We train our system from synthesized training data, but demonstrate its applicability to real-world data. Interestingly, our analysis shows that the information obtained from objects made out of multiple materials often is complementary and leads to better performance. |
| Tasks | |
| Published | 2016-11-28 |
| URL | http://arxiv.org/abs/1611.09325v2 |
| http://arxiv.org/pdf/1611.09325v2.pdf | |
| PWC | https://paperswithcode.com/paper/what-is-around-the-camera |
| Repo | |
| Framework | |
Feature Sensitive Label Fusion with Random Walker for Atlas-based Image Segmentation
| Title | Feature Sensitive Label Fusion with Random Walker for Atlas-based Image Segmentation |
| Authors | Siqi Bao, Albert C. S. Chung |
| Abstract | In this paper, a novel label fusion method is proposed for brain magnetic resonance image segmentation. This label fusion method is formulated on a graph, which embraces both label priors from atlases and anatomical priors from target image. To represent a pixel in a comprehensive way, three kinds of feature vectors are generated, including intensity, gradient and structural signature. To select candidate atlas nodes for fusion, rather than exact searching, randomized k-d tree with spatial constraint is introduced as an efficient approximation for high-dimensional feature matching. Feature Sensitive Label Prior (FSLP), which takes both the consistency and variety of different features into consideration, is proposed to gather atlas priors. As FSLP is a non-convex problem, one heuristic approach is further designed to solve it efficiently. Moreover, based on the anatomical knowledge, parts of the target pixels are also employed as graph seeds to assist the label fusion process and an iterative strategy is utilized to gradually update the label map. The comprehensive experiments carried out on two publicly available databases give results to demonstrate that the proposed method can obtain better segmentation quality. |
| Tasks | Semantic Segmentation |
| Published | 2016-10-24 |
| URL | http://arxiv.org/abs/1610.07475v2 |
| http://arxiv.org/pdf/1610.07475v2.pdf | |
| PWC | https://paperswithcode.com/paper/feature-sensitive-label-fusion-with-random |
| Repo | |
| Framework | |
