Paper Group ANR 392
Microscopic Muscle Image Enhancement. Compacting Neural Network Classifiers via Dropout Training. Learning and Transfer of Modulated Locomotor Controllers. Understanding and Optimizing the Performance of Distributed Machine Learning Applications on Apache Spark. Multilingual Word Embeddings using Multigraphs. The BioDynaMo Project. Exploring high-l …
Microscopic Muscle Image Enhancement
| Title | Microscopic Muscle Image Enhancement |
| Authors | Xiangfei Kong, Lin Yang |
| Abstract | We propose a robust image enhancement algorithm dedicated for muscle fiber specimen images captured by optical microscopes. Blur or out of focus problems are prevalent in muscle images during the image acquisition stage. Traditional image deconvolution methods do not work since they assume the blur kernels are known and also produce ring artifacts. We provide a compact framework which involves a novel spatially non-uniform blind deblurring approach specialized to muscle images which automatically detects and alleviates degraded regions. Ring artifacts problems are addressed and a kernel propagation strategy is proposed to speedup the algorithm and deals with the high non-uniformity of the blur kernels on muscle images. Experiments show that the proposed framework performs well on muscle images taken with modern advanced optical microscopes. Our framework is free of laborious parameter settings and is computationally efficient. |
| Tasks | Deblurring, Image Deconvolution, Image Enhancement |
| Published | 2016-12-17 |
| URL | http://arxiv.org/abs/1612.05719v1 |
| http://arxiv.org/pdf/1612.05719v1.pdf | |
| PWC | https://paperswithcode.com/paper/microscopic-muscle-image-enhancement |
| Repo | |
| Framework | |
Compacting Neural Network Classifiers via Dropout Training
| Title | Compacting Neural Network Classifiers via Dropout Training |
| Authors | Yotaro Kubo, George Tucker, Simon Wiesler |
| Abstract | We introduce dropout compaction, a novel method for training feed-forward neural networks which realizes the performance gains of training a large model with dropout regularization, yet extracts a compact neural network for run-time efficiency. In the proposed method, we introduce a sparsity-inducing prior on the per unit dropout retention probability so that the optimizer can effectively prune hidden units during training. By changing the prior hyperparameters, we can control the size of the resulting network. We performed a systematic comparison of dropout compaction and competing methods on several real-world speech recognition tasks and found that dropout compaction achieved comparable accuracy with fewer than 50% of the hidden units, translating to a 2.5x speedup in run-time. |
| Tasks | Speech Recognition |
| Published | 2016-11-18 |
| URL | http://arxiv.org/abs/1611.06148v2 |
| http://arxiv.org/pdf/1611.06148v2.pdf | |
| PWC | https://paperswithcode.com/paper/compacting-neural-network-classifiers-via |
| Repo | |
| Framework | |
Learning and Transfer of Modulated Locomotor Controllers
| Title | Learning and Transfer of Modulated Locomotor Controllers |
| Authors | Nicolas Heess, Greg Wayne, Yuval Tassa, Timothy Lillicrap, Martin Riedmiller, David Silver |
| Abstract | We study a novel architecture and training procedure for locomotion tasks. A high-frequency, low-level “spinal” network with access to proprioceptive sensors learns sensorimotor primitives by training on simple tasks. This pre-trained module is fixed and connected to a low-frequency, high-level “cortical” network, with access to all sensors, which drives behavior by modulating the inputs to the spinal network. Where a monolithic end-to-end architecture fails completely, learning with a pre-trained spinal module succeeds at multiple high-level tasks, and enables the effective exploration required to learn from sparse rewards. We test our proposed architecture on three simulated bodies: a 16-dimensional swimming snake, a 20-dimensional quadruped, and a 54-dimensional humanoid. Our results are illustrated in the accompanying video at https://youtu.be/sboPYvhpraQ |
| Tasks | |
| Published | 2016-10-17 |
| URL | http://arxiv.org/abs/1610.05182v1 |
| http://arxiv.org/pdf/1610.05182v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-and-transfer-of-modulated-locomotor |
| Repo | |
| Framework | |
Understanding and Optimizing the Performance of Distributed Machine Learning Applications on Apache Spark
| Title | Understanding and Optimizing the Performance of Distributed Machine Learning Applications on Apache Spark |
| Authors | Celestine Dünner, Thomas Parnell, Kubilay Atasu, Manolis Sifalakis, Haralampos Pozidis |
| Abstract | In this paper we explore the performance limits of Apache Spark for machine learning applications. We begin by analyzing the characteristics of a state-of-the-art distributed machine learning algorithm implemented in Spark and compare it to an equivalent reference implementation using the high performance computing framework MPI. We identify critical bottlenecks of the Spark framework and carefully study their implications on the performance of the algorithm. In order to improve Spark performance we then propose a number of practical techniques to alleviate some of its overheads. However, optimizing computational efficiency and framework related overheads is not the only key to performance – we demonstrate that in order to get the best performance out of any implementation it is necessary to carefully tune the algorithm to the respective trade-off between computation time and communication latency. The optimal trade-off depends on both the properties of the distributed algorithm as well as infrastructure and framework-related characteristics. Finally, we apply these technical and algorithmic optimizations to three different distributed linear machine learning algorithms that have been implemented in Spark. We present results using five large datasets and demonstrate that by using the proposed optimizations, we can achieve a reduction in the performance difference between Spark and MPI from 20x to 2x. |
| Tasks | |
| Published | 2016-12-05 |
| URL | http://arxiv.org/abs/1612.01437v2 |
| http://arxiv.org/pdf/1612.01437v2.pdf | |
| PWC | https://paperswithcode.com/paper/understanding-and-optimizing-the-performance |
| Repo | |
| Framework | |
Multilingual Word Embeddings using Multigraphs
| Title | Multilingual Word Embeddings using Multigraphs |
| Authors | Radu Soricut, Nan Ding |
| Abstract | We present a family of neural-network–inspired models for computing continuous word representations, specifically designed to exploit both monolingual and multilingual text. This framework allows us to perform unsupervised training of embeddings that exhibit higher accuracy on syntactic and semantic compositionality, as well as multilingual semantic similarity, compared to previous models trained in an unsupervised fashion. We also show that such multilingual embeddings, optimized for semantic similarity, can improve the performance of statistical machine translation with respect to how it handles words not present in the parallel data. |
| Tasks | Machine Translation, Multilingual Word Embeddings, Semantic Similarity, Semantic Textual Similarity, Word Embeddings |
| Published | 2016-12-14 |
| URL | http://arxiv.org/abs/1612.04732v1 |
| http://arxiv.org/pdf/1612.04732v1.pdf | |
| PWC | https://paperswithcode.com/paper/multilingual-word-embeddings-using |
| Repo | |
| Framework | |
The BioDynaMo Project
| Title | The BioDynaMo Project |
| Authors | Roman Bauer, Lukas Breitwieser, Alberto Di Meglio, Leonard Johard, Marcus Kaiser, Marco Manca, Manuel Mazzara, Max Talanov |
| Abstract | Computer simulations have become a very powerful tool for scientific research. Given the vast complexity that comes with many open scientific questions, a purely analytical or experimental approach is often not viable. For example, biological systems (such as the human brain) comprise an extremely complex organization and heterogeneous interactions across different spatial and temporal scales. In order to facilitate research on such problems, the BioDynaMo project (\url{https://biodynamo.web.cern.ch/}) aims at a general platform for computer simulations for biological research. Since the scientific investigations require extensive computer resources, this platform should be executable on hybrid cloud computing systems, allowing for the efficient use of state-of-the-art computing technology. This paper describes challenges during the early stages of the software development process. In particular, we describe issues regarding the implementation and the highly interdisciplinary as well as international nature of the collaboration. Moreover, we explain the methodologies, the approach, and the lessons learnt by the team during these first stages. |
| Tasks | |
| Published | 2016-07-10 |
| URL | http://arxiv.org/abs/1607.02717v1 |
| http://arxiv.org/pdf/1607.02717v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-biodynamo-project |
| Repo | |
| Framework | |
Exploring high-level Perspectives on Self-Configuration Capabilities of Systems
| Title | Exploring high-level Perspectives on Self-Configuration Capabilities of Systems |
| Authors | Aleksander Lodwich |
| Abstract | Optimization of product performance repetitively introduces the need to make products adaptive in a more general sense. This more general idea is often captured under the term ‘self-configuration’. Despite the importance of such capability, research work on this feature appears isolated by technical domains. It is not easy to tell quickly whether the approaches chosen in different technological domains introduce new ideas or whether the differences just reflect domain idiosyncrasies. For the sake of easy identification of key differences between systems with self-configuring capabilities, I will explore higher level concepts for understanding self-configuration, such as the {\Omega}-units, in order to provide theoretical instruments for connecting different areas of technology and research. |
| Tasks | |
| Published | 2016-06-28 |
| URL | http://arxiv.org/abs/1606.08906v1 |
| http://arxiv.org/pdf/1606.08906v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-high-level-perspectives-on-self |
| Repo | |
| Framework | |
Translation Quality Estimation using Recurrent Neural Network
| Title | Translation Quality Estimation using Recurrent Neural Network |
| Authors | Raj Nath Patel, Sasikumar M |
| Abstract | This paper describes our submission to the shared task on word/phrase level Quality Estimation (QE) in the First Conference on Statistical Machine Translation (WMT16). The objective of the shared task was to predict if the given word/phrase is a correct/incorrect (OK/BAD) translation in the given sentence. In this paper, we propose a novel approach for word level Quality Estimation using Recurrent Neural Network Language Model (RNN-LM) architecture. RNN-LMs have been found very effective in different Natural Language Processing (NLP) applications. RNN-LM is mainly used for vector space language modeling for different NLP problems. For this task, we modify the architecture of RNN-LM. The modified system predicts a label (OK/BAD) in the slot rather than predicting the word. The input to the system is a word sequence, similar to the standard RNN-LM. The approach is language independent and requires only the translated text for QE. To estimate the phrase level quality, we use the output of the word level QE system. |
| Tasks | Language Modelling, Machine Translation |
| Published | 2016-10-16 |
| URL | http://arxiv.org/abs/1610.04841v2 |
| http://arxiv.org/pdf/1610.04841v2.pdf | |
| PWC | https://paperswithcode.com/paper/translation-quality-estimation-using |
| Repo | |
| Framework | |
Census Signal Temporal Logic Inference for Multi-Agent Group Behavior Analysis
| Title | Census Signal Temporal Logic Inference for Multi-Agent Group Behavior Analysis |
| Authors | Zhe Xu, Agung Julius |
| Abstract | In this paper, we define a novel census signal temporal logic (CensusSTL) that focuses on the number of agents in different subsets of a group that complete a certain task specified by the signal temporal logic (STL). CensusSTL consists of an “inner logic” STL formula and an “outer logic” STL formula. We present a new inference algorithm to infer CensusSTL formulae from the trajectory data of a group of agents. We first identify the “inner logic” STL formula and then infer the subgroups based on whether the agents’ behaviors satisfy the “inner logic” formula at each time point. We use two different approaches to infer the subgroups based on similarity and complementarity, respectively. The “outer logic” CensusSTL formula is inferred from the census trajectories of different subgroups. We apply the algorithm in analyzing data from a soccer match by inferring the CensusSTL formula for different subgroups of a soccer team. |
| Tasks | |
| Published | 2016-10-05 |
| URL | http://arxiv.org/abs/1610.05612v1 |
| http://arxiv.org/pdf/1610.05612v1.pdf | |
| PWC | https://paperswithcode.com/paper/census-signal-temporal-logic-inference-for |
| Repo | |
| Framework | |
Learning from Multiway Data: Simple and Efficient Tensor Regression
| Title | Learning from Multiway Data: Simple and Efficient Tensor Regression |
| Authors | Rose Yu, Yan Liu |
| Abstract | Tensor regression has shown to be advantageous in learning tasks with multi-directional relatedness. Given massive multiway data, traditional methods are often too slow to operate on or suffer from memory bottleneck. In this paper, we introduce subsampled tensor projected gradient to solve the problem. Our algorithm is impressively simple and efficient. It is built upon projected gradient method with fast tensor power iterations, leveraging randomized sketching for further acceleration. Theoretical analysis shows that our algorithm converges to the correct solution in fixed number of iterations. The memory requirement grows linearly with the size of the problem. We demonstrate superior empirical performance on both multi-linear multi-task learning and spatio-temporal applications. |
| Tasks | Multi-Task Learning |
| Published | 2016-07-08 |
| URL | http://arxiv.org/abs/1607.02535v1 |
| http://arxiv.org/pdf/1607.02535v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-from-multiway-data-simple-and |
| Repo | |
| Framework | |
Numerically Grounded Language Models for Semantic Error Correction
| Title | Numerically Grounded Language Models for Semantic Error Correction |
| Authors | Georgios P. Spithourakis, Isabelle Augenstein, Sebastian Riedel |
| Abstract | Semantic error detection and correction is an important task for applications such as fact checking, speech-to-text or grammatical error correction. Current approaches generally focus on relatively shallow semantics and do not account for numeric quantities. Our approach uses language models grounded in numbers within the text. Such groundings are easily achieved for recurrent neural language model architectures, which can be further conditioned on incomplete background knowledge bases. Our evaluation on clinical reports shows that numerical grounding improves perplexity by 33% and F1 for semantic error correction by 5 points when compared to ungrounded approaches. Conditioning on a knowledge base yields further improvements. |
| Tasks | Grammatical Error Correction, Language Modelling |
| Published | 2016-08-14 |
| URL | http://arxiv.org/abs/1608.04147v1 |
| http://arxiv.org/pdf/1608.04147v1.pdf | |
| PWC | https://paperswithcode.com/paper/numerically-grounded-language-models-for |
| Repo | |
| Framework | |
Peak Criterion for Choosing Gaussian Kernel Bandwidth in Support Vector Data Description
| Title | Peak Criterion for Choosing Gaussian Kernel Bandwidth in Support Vector Data Description |
| Authors | Deovrat Kakde, Arin Chaudhuri, Seunghyun Kong, Maria Jahja, Hansi Jiang, Jorge Silva |
| Abstract | Support Vector Data Description (SVDD) is a machine-learning technique used for single class classification and outlier detection. SVDD formulation with kernel function provides a flexible boundary around data. The value of kernel function parameters affects the nature of the data boundary. For example, it is observed that with a Gaussian kernel, as the value of kernel bandwidth is lowered, the data boundary changes from spherical to wiggly. The spherical data boundary leads to underfitting, and an extremely wiggly data boundary leads to overfitting. In this paper, we propose empirical criterion to obtain good values of the Gaussian kernel bandwidth parameter. This criterion provides a smooth boundary that captures the essential geometric features of the data. |
| Tasks | Outlier Detection |
| Published | 2016-02-17 |
| URL | http://arxiv.org/abs/1602.05257v3 |
| http://arxiv.org/pdf/1602.05257v3.pdf | |
| PWC | https://paperswithcode.com/paper/peak-criterion-for-choosing-gaussian-kernel |
| Repo | |
| Framework | |
Clustering on the Edge: Learning Structure in Graphs
| Title | Clustering on the Edge: Learning Structure in Graphs |
| Authors | Matt Barnes, Artur Dubrawski |
| Abstract | With the recent popularity of graphical clustering methods, there has been an increased focus on the information between samples. We show how learning cluster structure using edge features naturally and simultaneously determines the most likely number of clusters and addresses data scale issues. These results are particularly useful in instances where (a) there are a large number of clusters and (b) we have some labeled edges. Applications in this domain include image segmentation, community discovery and entity resolution. Our model is an extension of the planted partition model and our solution uses results of correlation clustering, which achieves a partition O(log(n))-close to the log-likelihood of the true clustering. |
| Tasks | Entity Resolution, Semantic Segmentation |
| Published | 2016-05-05 |
| URL | http://arxiv.org/abs/1605.01779v1 |
| http://arxiv.org/pdf/1605.01779v1.pdf | |
| PWC | https://paperswithcode.com/paper/clustering-on-the-edge-learning-structure-in |
| Repo | |
| Framework | |
Superintelligence cannot be contained: Lessons from Computability Theory
| Title | Superintelligence cannot be contained: Lessons from Computability Theory |
| Authors | Manuel Alfonseca, Manuel Cebrian, Antonio Fernandez Anta, Lorenzo Coviello, Andres Abeliuk, Iyad Rahwan |
| Abstract | Superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. In light of recent advances in machine intelligence, a number of scientists, philosophers and technologists have revived the discussion about the potential catastrophic risks entailed by such an entity. In this article, we trace the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment. We argue that such containment is, in principle, impossible, due to fundamental limits inherent to computing itself. Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) infeasible. |
| Tasks | |
| Published | 2016-07-04 |
| URL | http://arxiv.org/abs/1607.00913v1 |
| http://arxiv.org/pdf/1607.00913v1.pdf | |
| PWC | https://paperswithcode.com/paper/superintelligence-cannot-be-contained-lessons |
| Repo | |
| Framework | |
Learning to detect and localize many objects from few examples
| Title | Learning to detect and localize many objects from few examples |
| Authors | Bastien Moysset, Christoper Kermorvant, Christian Wolf |
| Abstract | The current trend in object detection and localization is to learn predictions with high capacity deep neural networks trained on a very large amount of annotated data and using a high amount of processing power. In this work, we propose a new neural model which directly predicts bounding box coordinates. The particularity of our contribution lies in the local computations of predictions with a new form of local parameter sharing which keeps the overall amount of trainable parameters low. Key components of the model are spatial 2D-LSTM recurrent layers which convey contextual information between the regions of the image. We show that this model is more powerful than the state of the art in applications where training data is not as abundant as in the classical configuration of natural images and Imagenet/Pascal VOC tasks. We particularly target the detection of text in document images, but our method is not limited to this setting. The proposed model also facilitates the detection of many objects in a single image and can deal with inputs of variable sizes without resizing. |
| Tasks | Object Detection |
| Published | 2016-11-17 |
| URL | http://arxiv.org/abs/1611.05664v1 |
| http://arxiv.org/pdf/1611.05664v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-detect-and-localize-many-objects |
| Repo | |
| Framework | |
