Paper Group ANR 249
Model-driven Simulations for Deep Convolutional Neural Networks. Geometry-aware stationary subspace analysis. Spacetimes with Semantics (III) - The Structure of Functional Knowledge Representation and Artificial Reasoning. What is the right way to represent document images?. Learning-Based Resource Allocation Scheme for TDD-Based CRAN System. Class …
Model-driven Simulations for Deep Convolutional Neural Networks
| Title | Model-driven Simulations for Deep Convolutional Neural Networks |
| Authors | V S R Veeravasarapu, Constantin Rothkopf, Visvanathan Ramesh |
| Abstract | The use of simulated virtual environments to train deep convolutional neural networks (CNN) is a currently active practice to reduce the (real)data-hungriness of the deep CNN models, especially in application domains in which large scale real data and/or groundtruth acquisition is difficult or laborious. Recent approaches have attempted to harness the capabilities of existing video games, animated movies to provide training data with high precision groundtruth. However, a stumbling block is in how one can certify generalization of the learned models and their usefulness in real world data sets. This opens up fundamental questions such as: What is the role of photorealism of graphics simulations in training CNN models? Are the trained models valid in reality? What are possible ways to reduce the performance bias? In this work, we begin to address theses issues systematically in the context of urban semantic understanding with CNNs. Towards this end, we (a) propose a simple probabilistic urban scene model, (b) develop a parametric rendering tool to synthesize the data with groundtruth, followed by (c) a systematic exploration of the impact of level-of-realism on the generality of the trained CNN model to real world; and domain adaptation concepts to minimize the performance bias. |
| Tasks | Domain Adaptation |
| Published | 2016-05-31 |
| URL | http://arxiv.org/abs/1605.09582v1 |
| http://arxiv.org/pdf/1605.09582v1.pdf | |
| PWC | https://paperswithcode.com/paper/model-driven-simulations-for-deep |
| Repo | |
| Framework | |
Geometry-aware stationary subspace analysis
| Title | Geometry-aware stationary subspace analysis |
| Authors | Inbal Horev, Florian Yger, Masashi Sugiyama |
| Abstract | In many real-world applications data exhibits non-stationarity, i.e., its distribution changes over time. One approach to handling non-stationarity is to remove or minimize it before attempting to analyze the data. In the context of brain computer interface (BCI) data analysis this may be done by means of stationary subspace analysis (SSA). The classic SSA method finds a matrix that projects the data onto a stationary subspace by optimizing a cost function based on a matrix divergence. In this work we present an alternative method for SSA based on a symmetrized version of this matrix divergence. We show that this frames the problem in terms of distances between symmetric positive definite (SPD) matrices, suggesting a geometric interpretation of the problem. Stemming from this geometric viewpoint, we introduce and analyze a method which utilizes the geometry of the SPD matrix manifold and the invariance properties of its metrics. Most notably we show that these invariances alleviate the need to whiten the input matrices, a common step in many SSA methods which often introduces errors. We demonstrate the usefulness of our technique in experiments on both synthesized and real-world data. |
| Tasks | |
| Published | 2016-05-25 |
| URL | http://arxiv.org/abs/1605.07785v1 |
| http://arxiv.org/pdf/1605.07785v1.pdf | |
| PWC | https://paperswithcode.com/paper/geometry-aware-stationary-subspace-analysis |
| Repo | |
| Framework | |
Spacetimes with Semantics (III) - The Structure of Functional Knowledge Representation and Artificial Reasoning
| Title | Spacetimes with Semantics (III) - The Structure of Functional Knowledge Representation and Artificial Reasoning |
| Authors | Mark Burgess |
| Abstract | Using the previously developed concepts of semantic spacetime, I explore the interpretation of knowledge representations, and their structure, as a semantic system, within the framework of promise theory. By assigning interpretations to phenomena, from observers to observed, we may approach a simple description of knowledge-based functional systems, with direct practical utility. The focus is especially on the interpretation of concepts, associative knowledge, and context awareness. The inference seems to be that most if not all of these concepts emerge from purely semantic spacetime properties, which opens the possibility for a more generalized understanding of what constitutes a learning, or even intelligent' system. Some key principles emerge for effective knowledge representation: 1) separation of spacetime scales, 2) the recurrence of four irreducible types of association, by which intent propagates: aggregation, causation, cooperation, and similarity, 3) the need for discrimination of identities (discrete), which is assisted by distinguishing timeline simultaneity from sequential events, and 4) the ability to learn (memory). It is at least plausible that emergent knowledge abstraction capabilities have their origin in basic spacetime structures. These notes present a unified view of mostly well-known results; they allow us to see information models, knowledge representations, machine learning, and semantic networking (transport and information base) in a common framework. The notion of smart spaces’ thus encompasses artificial systems as well as living systems, across many different scales, e.g. smart cities and organizations. |
| Tasks | |
| Published | 2016-08-07 |
| URL | http://arxiv.org/abs/1608.02193v4 |
| http://arxiv.org/pdf/1608.02193v4.pdf | |
| PWC | https://paperswithcode.com/paper/spacetimes-with-semantics-iii-the-structure |
| Repo | |
| Framework | |
What is the right way to represent document images?
| Title | What is the right way to represent document images? |
| Authors | Gabriela Csurka, Diane Larlus, Albert Gordo, Jon Almazan |
| Abstract | In this article we study the problem of document image representation based on visual features. We propose a comprehensive experimental study that compares three types of visual document image representations: (1) traditional so-called shallow features, such as the RunLength and the Fisher-Vector descriptors, (2) deep features based on Convolutional Neural Networks, and (3) features extracted from hybrid architectures that take inspiration from the two previous ones. We evaluate these features in several tasks (i.e. classification, clustering, and retrieval) and in different setups (e.g. domain transfer) using several public and in-house datasets. Our results show that deep features generally outperform other types of features when there is no domain shift and the new task is closely related to the one used to train the model. However, when a large domain or task shift is present, the Fisher-Vector shallow features generalize better and often obtain the best results. |
| Tasks | |
| Published | 2016-03-03 |
| URL | http://arxiv.org/abs/1603.01076v3 |
| http://arxiv.org/pdf/1603.01076v3.pdf | |
| PWC | https://paperswithcode.com/paper/what-is-the-right-way-to-represent-document |
| Repo | |
| Framework | |
Learning-Based Resource Allocation Scheme for TDD-Based CRAN System
| Title | Learning-Based Resource Allocation Scheme for TDD-Based CRAN System |
| Authors | Sahar Imtiaz, Hadi Ghauch, M. Mahboob Ur Rahman, George Koudouridis, James Gross |
| Abstract | Explosive growth in the use of smart wireless devices has necessitated the provision of higher data rates and always-on connectivity, which are the main motivators for designing the fifth generation (5G) systems. To achieve higher system efficiency, massive antenna deployment with tight coordination is one potential strategy for designing 5G systems, but has two types of associated system overhead. First is the synchronization overhead, which can be reduced by implementing a cloud radio access network (CRAN)-based architecture design, that separates the baseband processing and radio access functionality to achieve better system synchronization. Second is the overhead for acquiring channel state information (CSI) of the users present in the system, which, however, increases tremendously when instantaneous CSI is used to serve high-mobility users. To serve a large number of users, a CRAN system with a dense deployment of remote radio heads (RRHs) is considered, such that each user has a line-of-sight (LOS) link with the corresponding RRH. Since, the trajectory of movement for high-mobility users is predictable; therefore, fairly accurate position estimates for those users can be obtained, and can be used for resource allocation to serve the considered users. The resource allocation is dependent upon various correlated system parameters, and these correlations can be learned using well-known \emph{machine learning} algorithms. This paper proposes a novel \emph{learning-based resource allocation scheme} for time division duplex (TDD) based 5G CRAN systems with dense RRH deployment, by using only the users’ position estimates for resource allocation, thus avoiding the need for CSI acquisition. This reduces the overall system overhead significantly, while still achieving near-optimal system performance; thus, better (effective) system efficiency is achieved. (See the paper for full abstract) |
| Tasks | |
| Published | 2016-08-29 |
| URL | http://arxiv.org/abs/1608.07949v1 |
| http://arxiv.org/pdf/1608.07949v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-based-resource-allocation-scheme-for |
| Repo | |
| Framework | |
Classification and Verification of Online Handwritten Signatures with Time Causal Information Theory Quantifiers
| Title | Classification and Verification of Online Handwritten Signatures with Time Causal Information Theory Quantifiers |
| Authors | Osvaldo A. Rosso, Raydonal Ospina, Alejandro C. Frery |
| Abstract | We present a new approach for online handwritten signature classification and verification based on descriptors stemming from Information Theory. The proposal uses the Shannon Entropy, the Statistical Complexity, and the Fisher Information evaluated over the Bandt and Pompe symbolization of the horizontal and vertical coordinates of signatures. These six features are easy and fast to compute, and they are the input to an One-Class Support Vector Machine classifier. The results produced surpass state-of-the-art techniques that employ higher-dimensional feature spaces which often require specialized software and hardware. We assess the consistency of our proposal with respect to the size of the training sample, and we also use it to classify the signatures into meaningful groups. |
| Tasks | |
| Published | 2016-01-26 |
| URL | http://arxiv.org/abs/1601.06925v1 |
| http://arxiv.org/pdf/1601.06925v1.pdf | |
| PWC | https://paperswithcode.com/paper/classification-and-verification-of-online |
| Repo | |
| Framework | |
Stitching Stabilizer: Two-frame-stitching Video Stabilization for Embedded Systems
| Title | Stitching Stabilizer: Two-frame-stitching Video Stabilization for Embedded Systems |
| Authors | Masaki Satoh |
| Abstract | In conventional electronic video stabilization, the stabilized frame is obtained by cropping the input frame to cancel camera shake. While a small cropping size results in strong stabilization, it does not provide us satisfactory results from the viewpoint of image quality, because it narrows the angle of view. By fusing several frames, we can effectively expand the area of input frames, and achieve strong stabilization even with a large cropping size. Several methods for doing so have been studied. However, their computational costs are too high for embedded systems such as smartphones. We propose a simple, yet surprisingly effective algorithm, called the stitching stabilizer. It stitches only two frames together with a minimal computational cost. It can achieve real-time processes in embedded systems, for Full HD and 30 FPS videos. To clearly show the effect, we apply it to hyperlapse. Using several clips, we show it produces more strongly stabilized and natural results than the existing solutions from Microsoft and Instagram. |
| Tasks | |
| Published | 2016-03-22 |
| URL | http://arxiv.org/abs/1603.06678v1 |
| http://arxiv.org/pdf/1603.06678v1.pdf | |
| PWC | https://paperswithcode.com/paper/stitching-stabilizer-two-frame-stitching |
| Repo | |
| Framework | |
Comparison of Brain Networks with Unknown Correspondences
| Title | Comparison of Brain Networks with Unknown Correspondences |
| Authors | Sofia Ira Ktena, Sarah Parisot, Jonathan Passerat-Palmbach, Daniel Rueckert |
| Abstract | Graph theory has drawn a lot of attention in the field of Neuroscience during the last decade, mainly due to the abundance of tools that it provides to explore the interactions of elements in a complex network like the brain. The local and global organization of a brain network can shed light on mechanisms of complex cognitive functions, while disruptions within the network can be linked to neurodevelopmental disorders. In this effort, the construction of a representative brain network for each individual is critical for further analysis. Additionally, graph comparison is an essential step for inference and classification analyses on brain graphs. In this work we explore a method based on graph edit distance for evaluating graph similarity, when correspondences between network elements are unknown due to different underlying subdivisions of the brain. We test this method on 30 unrelated subjects as well as 40 twin pairs and show that this method can accurately reflect the higher similarity between two related networks compared to unrelated ones, while identifying node correspondences. |
| Tasks | Graph Similarity |
| Published | 2016-11-15 |
| URL | http://arxiv.org/abs/1611.04783v1 |
| http://arxiv.org/pdf/1611.04783v1.pdf | |
| PWC | https://paperswithcode.com/paper/comparison-of-brain-networks-with-unknown |
| Repo | |
| Framework | |
Learning from Conditional Distributions via Dual Embeddings
| Title | Learning from Conditional Distributions via Dual Embeddings |
| Authors | Bo Dai, Niao He, Yunpeng Pan, Byron Boots, Le Song |
| Abstract | Many machine learning tasks, such as learning with invariance and policy evaluation in reinforcement learning, can be characterized as problems of learning from conditional distributions. In such problems, each sample $x$ itself is associated with a conditional distribution $p(zx)$ represented by samples ${z_i}_{i=1}^M$, and the goal is to learn a function $f$ that links these conditional distributions to target values $y$. These learning problems become very challenging when we only have limited samples or in the extreme case only one sample from each conditional distribution. Commonly used approaches either assume that $z$ is independent of $x$, or require an overwhelmingly large samples from each conditional distribution. To address these challenges, we propose a novel approach which employs a new min-max reformulation of the learning from conditional distribution problem. With such new reformulation, we only need to deal with the joint distribution $p(z,x)$. We also design an efficient learning algorithm, Embedding-SGD, and establish theoretical sample complexity for such problems. Finally, our numerical experiments on both synthetic and real-world datasets show that the proposed approach can significantly improve over the existing algorithms. |
| Tasks | |
| Published | 2016-07-15 |
| URL | http://arxiv.org/abs/1607.04579v2 |
| http://arxiv.org/pdf/1607.04579v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-from-conditional-distributions-via |
| Repo | |
| Framework | |
Online Data Thinning via Multi-Subspace Tracking
| Title | Online Data Thinning via Multi-Subspace Tracking |
| Authors | Xin Jiang, Rebecca Willett |
| Abstract | In an era of ubiquitous large-scale streaming data, the availability of data far exceeds the capacity of expert human analysts. In many settings, such data is either discarded or stored unprocessed in datacenters. This paper proposes a method of online data thinning, in which large-scale streaming datasets are winnowed to preserve unique, anomalous, or salient elements for timely expert analysis. At the heart of this proposed approach is an online anomaly detection method based on dynamic, low-rank Gaussian mixture models. Specifically, the high-dimensional covariances matrices associated with the Gaussian components are associated with low-rank models. According to this model, most observations lie near a union of subspaces. The low-rank modeling mitigates the curse of dimensionality associated with anomaly detection for high-dimensional data, and recent advances in subspace clustering and subspace tracking allow the proposed method to adapt to dynamic environments. Furthermore, the proposed method allows subsampling, is robust to missing data, and uses a mini-batch online optimization approach. The resulting algorithms are scalable, efficient, and are capable of operating in real time. Experiments on wide-area motion imagery and e-mail databases illustrate the efficacy of the proposed approach. |
| Tasks | Anomaly Detection |
| Published | 2016-09-12 |
| URL | http://arxiv.org/abs/1609.03544v1 |
| http://arxiv.org/pdf/1609.03544v1.pdf | |
| PWC | https://paperswithcode.com/paper/online-data-thinning-via-multi-subspace |
| Repo | |
| Framework | |
The deterministic information bottleneck
| Title | The deterministic information bottleneck |
| Authors | DJ Strouse, David J Schwab |
| Abstract | Lossy compression and clustering fundamentally involve a decision about what features are relevant and which are not. The information bottleneck method (IB) by Tishby, Pereira, and Bialek formalized this notion as an information-theoretic optimization problem and proposed an optimal tradeoff between throwing away as many bits as possible, and selectively keeping those that are most important. In the IB, compression is measure my mutual information. Here, we introduce an alternative formulation that replaces mutual information with entropy, which we call the deterministic information bottleneck (DIB), that we argue better captures this notion of compression. As suggested by its name, the solution to the DIB problem turns out to be a deterministic encoder, or hard clustering, as opposed to the stochastic encoder, or soft clustering, that is optimal under the IB. We compare the IB and DIB on synthetic data, showing that the IB and DIB perform similarly in terms of the IB cost function, but that the DIB significantly outperforms the IB in terms of the DIB cost function. We also empirically find that the DIB offers a considerable gain in computational efficiency over the IB, over a range of convergence parameters. Our derivation of the DIB also suggests a method for continuously interpolating between the soft clustering of the IB and the hard clustering of the DIB. |
| Tasks | |
| Published | 2016-04-01 |
| URL | http://arxiv.org/abs/1604.00268v2 |
| http://arxiv.org/pdf/1604.00268v2.pdf | |
| PWC | https://paperswithcode.com/paper/the-deterministic-information-bottleneck |
| Repo | |
| Framework | |
A Primer on Coordinate Descent Algorithms
| Title | A Primer on Coordinate Descent Algorithms |
| Authors | Hao-Jun Michael Shi, Shenyinying Tu, Yangyang Xu, Wotao Yin |
| Abstract | This monograph presents a class of algorithms called coordinate descent algorithms for mathematicians, statisticians, and engineers outside the field of optimization. This particular class of algorithms has recently gained popularity due to their effectiveness in solving large-scale optimization problems in machine learning, compressed sensing, image processing, and computational statistics. Coordinate descent algorithms solve optimization problems by successively minimizing along each coordinate or coordinate hyperplane, which is ideal for parallelized and distributed computing. Avoiding detailed technicalities and proofs, this monograph gives relevant theory and examples for practitioners to effectively apply coordinate descent to modern problems in data science and engineering. |
| Tasks | |
| Published | 2016-09-30 |
| URL | http://arxiv.org/abs/1610.00040v2 |
| http://arxiv.org/pdf/1610.00040v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-primer-on-coordinate-descent-algorithms |
| Repo | |
| Framework | |
The Controlled Natural Language of Randall Munroe’s Thing Explainer
| Title | The Controlled Natural Language of Randall Munroe’s Thing Explainer |
| Authors | Tobias Kuhn |
| Abstract | It is rare that texts or entire books written in a Controlled Natural Language (CNL) become very popular, but exactly this has happened with a book that has been published last year. Randall Munroe’s Thing Explainer uses only the 1’000 most often used words of the English language together with drawn pictures to explain complicated things such as nuclear reactors, jet engines, the solar system, and dishwashers. This restricted language is a very interesting new case for the CNL community. I describe here its place in the context of existing approaches on Controlled Natural Languages, and I provide a first analysis from a scientific perspective, covering the word production rules and word distributions. |
| Tasks | |
| Published | 2016-05-09 |
| URL | http://arxiv.org/abs/1605.02457v1 |
| http://arxiv.org/pdf/1605.02457v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-controlled-natural-language-of-randall |
| Repo | |
| Framework | |
Evaluating Automatic Speech Recognition Systems in Comparison With Human Perception Results Using Distinctive Feature Measures
| Title | Evaluating Automatic Speech Recognition Systems in Comparison With Human Perception Results Using Distinctive Feature Measures |
| Authors | Xiang Kong, Jeung-Yoon Choi, Stefanie Shattuck-Hufnagel |
| Abstract | This paper describes methods for evaluating automatic speech recognition (ASR) systems in comparison with human perception results, using measures derived from linguistic distinctive features. Error patterns in terms of manner, place and voicing are presented, along with an examination of confusion matrices via a distinctive-feature-distance metric. These evaluation methods contrast with conventional performance criteria that focus on the phone or word level, and are intended to provide a more detailed profile of ASR system performance,as well as a means for direct comparison with human perception results at the sub-phonemic level. |
| Tasks | Speech Recognition |
| Published | 2016-12-13 |
| URL | http://arxiv.org/abs/1612.03990v1 |
| http://arxiv.org/pdf/1612.03990v1.pdf | |
| PWC | https://paperswithcode.com/paper/evaluating-automatic-speech-recognition |
| Repo | |
| Framework | |
Adaptive Geometric Multiscale Approximations for Intrinsically Low-dimensional Data
| Title | Adaptive Geometric Multiscale Approximations for Intrinsically Low-dimensional Data |
| Authors | Wenjing Liao, Mauro Maggioni |
| Abstract | We consider the problem of efficiently approximating and encoding high-dimensional data sampled from a probability distribution $\rho$ in $\mathbb{R}^D$, that is nearly supported on a $d$-dimensional set $\mathcal{M}$ - for example supported on a $d$-dimensional Riemannian manifold. Geometric Multi-Resolution Analysis (GMRA) provides a robust and computationally efficient procedure to construct low-dimensional geometric approximations of $\mathcal{M}$ at varying resolutions. We introduce a thresholding algorithm on the geometric wavelet coefficients, leading to what we call adaptive GMRA approximations. We show that these data-driven, empirical approximations perform well, when the threshold is chosen as a suitable universal function of the number of samples $n$, on a wide variety of measures $\rho$, that are allowed to exhibit different regularity at different scales and locations, thereby efficiently encoding data from more complex measures than those supported on manifolds. These approximations yield a data-driven dictionary, together with a fast transform mapping data to coefficients, and an inverse of such a map. The algorithms for both the dictionary construction and the transforms have complexity $C n \log n$ with the constant linear in $D$ and exponential in $d$. Our work therefore establishes adaptive GMRA as a fast dictionary learning algorithm with approximation guarantees. We include several numerical experiments on both synthetic and real data, confirming our theoretical results and demonstrating the effectiveness of adaptive GMRA. |
| Tasks | Dictionary Learning |
| Published | 2016-11-03 |
| URL | http://arxiv.org/abs/1611.01179v2 |
| http://arxiv.org/pdf/1611.01179v2.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-geometric-multiscale-approximations |
| Repo | |
| Framework | |
