July 28, 2019

3322 words 16 mins read

Paper Group ANR 381

Learning Multi-level Region Consistency with Dense Multi-label Networks for Semantic Segmentation. A Simple LSTM model for Transition-based Dependency Parsing. Encoding Multi-Resolution Brain Networks Using Unsupervised Deep Learning. An Empirical Study of the Effects of Spurious Transitions on Abstraction-based Heuristics. An Adaptive Fuzzy-Based …

Learning Multi-level Region Consistency with Dense Multi-label Networks for Semantic Segmentation


Title	Learning Multi-level Region Consistency with Dense Multi-label Networks for Semantic Segmentation
Authors	Tong Shen, Guosheng Lin, Chunhua Shen, Ian Reid
Abstract	Semantic image segmentation is a fundamental task in image understanding. Per-pixel semantic labelling of an image benefits greatly from the ability to consider region consistency both locally and globally. However, many Fully Convolutional Network based methods do not impose such consistency, which may give rise to noisy and implausible predictions. We address this issue by proposing a dense multi-label network module that is able to encourage the region consistency at different levels. This simple but effective module can be easily integrated into any semantic segmentation systems. With comprehensive experiments, we show that the dense multi-label can successfully remove the implausible labels and clear the confusion so as to boost the performance of semantic segmentation systems.
Tasks	Semantic Segmentation
Published	2017-01-25
URL	http://arxiv.org/abs/1701.07122v1
PDF	http://arxiv.org/pdf/1701.07122v1.pdf
PWC	https://paperswithcode.com/paper/learning-multi-level-region-consistency-with
Repo
Framework

A Simple LSTM model for Transition-based Dependency Parsing


Title	A Simple LSTM model for Transition-based Dependency Parsing
Authors	Mohab Elkaref, Bernd Bohnet
Abstract	We present a simple LSTM-based transition-based dependency parser. Our model is composed of a single LSTM hidden layer replacing the hidden layer in the usual feed-forward network architecture. We also propose a new initialization method that uses the pre-trained weights from a feed-forward neural network to initialize our LSTM-based model. We also show that using dropout on the input layer has a positive effect on performance. Our final parser achieves a 93.06% unlabeled and 91.01% labeled attachment score on the Penn Treebank. We additionally replace LSTMs with GRUs and Elman units in our model and explore the effectiveness of our initialization method on individual gates constituting all three types of RNN units.
Tasks	Dependency Parsing, Transition-Based Dependency Parsing
Published	2017-08-29
URL	http://arxiv.org/abs/1708.08959v2
PDF	http://arxiv.org/pdf/1708.08959v2.pdf
PWC	https://paperswithcode.com/paper/a-simple-lstm-model-for-transition-based
Repo
Framework

Encoding Multi-Resolution Brain Networks Using Unsupervised Deep Learning


Title	Encoding Multi-Resolution Brain Networks Using Unsupervised Deep Learning
Authors	Arash Rahnama, Abdullah Alchihabi, Vijay Gupta, Panos Antsaklis, Fatos T. Yarman Vural
Abstract	The main goal of this study is to extract a set of brain networks in multiple time-resolutions to analyze the connectivity patterns among the anatomic regions for a given cognitive task. We suggest a deep architecture which learns the natural groupings of the connectivity patterns of human brain in multiple time-resolutions. The suggested architecture is tested on task data set of Human Connectome Project (HCP) where we extract multi-resolution networks, each of which corresponds to a cognitive task. At the first level of this architecture, we decompose the fMRI signal into multiple sub-bands using wavelet decompositions. At the second level, for each sub-band, we estimate a brain network extracted from short time windows of the fMRI signal. At the third level, we feed the adjacency matrices of each mesh network at each time-resolution into an unsupervised deep learning algorithm, namely, a Stacked De- noising Auto-Encoder (SDAE). The outputs of the SDAE provide a compact connectivity representation for each time window at each sub-band of the fMRI signal. We concatenate the learned representations of all sub-bands at each window and cluster them by a hierarchical algorithm to find the natural groupings among the windows. We observe that each cluster represents a cognitive task with a performance of 93% Rand Index and 71% Adjusted Rand Index. We visualize the mean values and the precisions of the networks at each component of the cluster mixture. The mean brain networks at cluster centers show the variations among cognitive tasks and the precision of each cluster shows the within cluster variability of networks, across the subjects.
Tasks
Published	2017-08-13
URL	http://arxiv.org/abs/1708.04232v1
PDF	http://arxiv.org/pdf/1708.04232v1.pdf
PWC	https://paperswithcode.com/paper/encoding-multi-resolution-brain-networks
Repo
Framework

An Empirical Study of the Effects of Spurious Transitions on Abstraction-based Heuristics


Title	An Empirical Study of the Effects of Spurious Transitions on Abstraction-based Heuristics
Authors	Mehdi Sadeqi, Robert C. Holte, Sandra Zilles
Abstract	The efficient solution of state space search problems is often attempted by guiding search algorithms with heuristics (estimates of the distance from any state to the goal). A popular way for creating heuristic functions is by using an abstract version of the state space. However, the quality of abstraction-based heuristic functions, and thus the speed of search, can suffer from spurious transitions, i.e., state transitions in the abstract state space for which no corresponding transitions in the reachable component of the original state space exist. Our first contribution is a quantitative study demonstrating that the harmful effects of spurious transitions on heuristic functions can be substantial, in terms of both the increase in the number of abstract states and the decrease in the heuristic values, which may slow down search. Our second contribution is an empirical study on the benefits of removing a certain kind of spurious transition, namely those that involve states with a pair of mutually exclusive (mutex) variablevalue assignments. In the context of state space planning, a mutex pair is a pair of variable-value assignments that does not occur in any reachable state. Detecting mutex pairs is a problem that has been addressed frequently in the planning literature. Our study shows that there are cases in which mutex detection helps to eliminate harmful spurious transitions to a large extent and thus to speed up search substantially.
Tasks
Published	2017-11-14
URL	http://arxiv.org/abs/1711.05105v1
PDF	http://arxiv.org/pdf/1711.05105v1.pdf
PWC	https://paperswithcode.com/paper/an-empirical-study-of-the-effects-of-spurious
Repo
Framework

An Adaptive Fuzzy-Based System to Simulate, Quantify and Compensate Color Blindness


Title	An Adaptive Fuzzy-Based System to Simulate, Quantify and Compensate Color Blindness
Authors	Jinmi Lee, Wellington Pinheiro dos Santos
Abstract	About 8% of the male population of the world are affected by a determined type of color vision disturbance, which varies from the partial to complete reduction of the ability to distinguish certain colors. A considerable amount of color blind people are able to live all life long without knowing they have color vision disabilities and abnormalities. Nowadays the evolution of information technology and computer science, specifically image processing techniques and computer graphics, can be fundamental to aid at the development of adaptive color blindness correction tools. This paper presents a software tool based on Fuzzy Logic to evaluate the type and the degree of color blindness a person suffer from. In order to model several degrees of color blindness, herein this work we modified the classical linear transform-based simulation method by the use of fuzzy parameters. We also proposed four new methods to correct color blindness based on a fuzzy approach: Methods A and B, with and without histogram equalization. All the methods are based on combinations of linear transforms and histogram operations. In order to evaluate the results we implemented a web-based survey to get the best results according to optimize to distinguish different elements in an image. Results obtained from 40 volunteers proved that the Method B with histogram equalization got the best results for about 47% of volunteers.
Tasks
Published	2017-11-29
URL	http://arxiv.org/abs/1711.10662v1
PDF	http://arxiv.org/pdf/1711.10662v1.pdf
PWC	https://paperswithcode.com/paper/an-adaptive-fuzzy-based-system-to-simulate
Repo
Framework

Using Human Brain Activity to Guide Machine Learning


Title	Using Human Brain Activity to Guide Machine Learning
Authors	Ruth Fong, Walter Scheirer, David Cox
Abstract	Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. While the human brain has long served as a source of inspiration for machine learning, little effort has been made to directly use data collected from working brains as a guide for machine learning algorithms. Here we demonstrate a new paradigm of “neurally-weighted” machine learning, which takes fMRI measurements of human brain activity from subjects viewing images, and infuses these data into the training process of an object recognition learning algorithm to make it more consistent with the human brain. After training, these neurally-weighted classifiers are able to classify images without requiring any additional neural data. We show that our neural-weighting approach can lead to large performance gains when used with traditional machine vision features, as well as to significant improvements with already high-performing convolutional neural network features. The effectiveness of this approach points to a path forward for a new class of hybrid machine learning algorithms which take both inspiration and direct constraints from neuronal data.
Tasks	Object Recognition
Published	2017-03-16
URL	http://arxiv.org/abs/1703.05463v2
PDF	http://arxiv.org/pdf/1703.05463v2.pdf
PWC	https://paperswithcode.com/paper/using-human-brain-activity-to-guide-machine
Repo
Framework

Fast and Scalable Distributed Deep Convolutional Autoencoder for fMRI Big Data Analytics


Title	Fast and Scalable Distributed Deep Convolutional Autoencoder for fMRI Big Data Analytics
Authors	Milad Makkie, Heng Huang, Yu Zhao, Athanasios V. Vasilakos, Tianming Liu
Abstract	In recent years, analyzing task-based fMRI (tfMRI) data has become an essential tool for understanding brain function and networks. However, due to the sheer size of tfMRI data, its intrinsic complex structure, and lack of ground truth of underlying neural activities, modeling tfMRI data is hard and challenging. Previously proposed data-modeling methods including Independent Component Analysis (ICA) and Sparse Dictionary Learning only provided a weakly established model based on blind source separation under the strong assumption that original fMRI signals could be linearly decomposed into time series components with corresponding spatial maps. Meanwhile, analyzing and learning a large amount of tfMRI data from a variety of subjects has been shown to be very demanding but yet challenging even with technological advances in computational hardware. Given the Convolutional Neural Network (CNN), a robust method for learning high-level abstractions from low-level data such as tfMRI time series, in this work we propose a fast and scalable novel framework for distributed deep Convolutional Autoencoder model. This model aims to both learn the complex hierarchical structure of the tfMRI data and to leverage the processing power of multiple GPUs in a distributed fashion. To implement such a model, we have created an enhanced processing pipeline on the top of Apache Spark and Tensorflow library, leveraging from a very large cluster of GPU machines. Experimental data from applying the model on the Human Connectome Project (HCP) show that the proposed model is efficient and scalable toward tfMRI big data analytics, thus enabling data-driven extraction of hierarchical neuroscientific information from massive fMRI big data in the future.
Tasks	Dictionary Learning, Time Series
Published	2017-10-24
URL	http://arxiv.org/abs/1710.08961v3
PDF	http://arxiv.org/pdf/1710.08961v3.pdf
PWC	https://paperswithcode.com/paper/fast-and-scalable-distributed-deep
Repo
Framework

A Sequence-Based Mesh Classifier for the Prediction of Protein-Protein Interactions


Title	A Sequence-Based Mesh Classifier for the Prediction of Protein-Protein Interactions
Authors	Edgar D. Coelho, Igor N. Cruz, André Santiago, José Luis Oliveira, António Dourado, Joel P. Arrais
Abstract	The worldwide surge of multiresistant microbial strains has propelled the search for alternative treatment options. The study of Protein-Protein Interactions (PPIs) has been a cornerstone in the clarification of complex physiological and pathogenic processes, thus being a priority for the identification of vital components and mechanisms in pathogens. Despite the advances of laboratorial techniques, computational models allow the screening of protein interactions between entire proteomes in a fast and inexpensive manner. Here, we present a supervised machine learning model for the prediction of PPIs based on the protein sequence. We cluster amino acids regarding their physicochemical properties, and use the discrete cosine transform to represent protein sequences. A mesh of classifiers was constructed to create hyper-specialised classifiers dedicated to the most relevant pairs of molecular function annotations from Gene Ontology. Based on an exhaustive evaluation that includes datasets with different configurations, cross-validation and out-of-sampling validation, the obtained results outscore the state-of-the-art for sequence-based methods. For the final mesh model using SVM with RBF, a consistent average AUC of 0.84 was attained.
Tasks
Published	2017-11-12
URL	http://arxiv.org/abs/1711.04294v1
PDF	http://arxiv.org/pdf/1711.04294v1.pdf
PWC	https://paperswithcode.com/paper/a-sequence-based-mesh-classifier-for-the
Repo
Framework

Adversarial Multi-Criteria Learning for Chinese Word Segmentation


Title	Adversarial Multi-Criteria Learning for Chinese Word Segmentation
Authors	Xinchi Chen, Zhan Shi, Xipeng Qiu, Xuanjing Huang
Abstract	Different linguistic perspectives causes many diverse segmentation criteria for Chinese word segmentation (CWS). Most existing methods focus on improve the performance for each single criterion. However, it is interesting to exploit these different criteria and mining their common underlying knowledge. In this paper, we propose adversarial multi-criteria learning for CWS by integrating shared knowledge from multiple heterogeneous segmentation criteria. Experiments on eight corpora with heterogeneous segmentation criteria show that the performance of each corpus obtains a significant improvement, compared to single-criterion learning. Source codes of this paper are available on Github.
Tasks	Chinese Word Segmentation
Published	2017-04-25
URL	http://arxiv.org/abs/1704.07556v1
PDF	http://arxiv.org/pdf/1704.07556v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-multi-criteria-learning-for
Repo
Framework

Gaussian Process Neurons Learn Stochastic Activation Functions


Title	Gaussian Process Neurons Learn Stochastic Activation Functions
Authors	Sebastian Urban, Marcus Basalla, Patrick van der Smagt
Abstract	We propose stochastic, non-parametric activation functions that are fully learnable and individual to each neuron. Complexity and the risk of overfitting are controlled by placing a Gaussian process prior over these functions. The result is the Gaussian process neuron, a probabilistic unit that can be used as the basic building block for probabilistic graphical models that resemble the structure of neural networks. The proposed model can intrinsically handle uncertainties in its inputs and self-estimate the confidence of its predictions. Using variational Bayesian inference and the central limit theorem, a fully deterministic loss function is derived, allowing it to be trained as efficiently as a conventional neural network using mini-batch gradient descent. The posterior distribution of activation functions is inferred from the training data alongside the weights of the network. The proposed model favorably compares to deep Gaussian processes, both in model complexity and efficiency of inference. It can be directly applied to recurrent or convolutional network structures, allowing its use in audio and image processing tasks. As an preliminary empirical evaluation we present experiments on regression and classification tasks, in which our model achieves performance comparable to or better than a Dropout regularized neural network with a fixed activation function. Experiments are ongoing and results will be added as they become available.
Tasks	Bayesian Inference, Gaussian Processes
Published	2017-11-29
URL	http://arxiv.org/abs/1711.11059v1
PDF	http://arxiv.org/pdf/1711.11059v1.pdf
PWC	https://paperswithcode.com/paper/gaussian-process-neurons-learn-stochastic
Repo
Framework

Protecting Genomic Privacy by a Sequence-Similarity Based Obfuscation Method


Title	Protecting Genomic Privacy by a Sequence-Similarity Based Obfuscation Method
Authors	Shibiao Wan, Man-Wai Mak, Sun-Yuan Kung
Abstract	In the post-genomic era, large-scale personal DNA sequences are produced and collected for genetic medical diagnoses and new drug discovery, which, however, simultaneously poses serious challenges to the protection of personal genomic privacy. Existing genomic privacy-protection methods are either time-consuming or with low accuracy. To tackle these problems, this paper proposes a sequence similarity-based obfuscation method, namely IterMegaBLAST, for fast and reliable protection of personal genomic privacy. Specifically, given a randomly selected sequence from a dataset of DNA sequences, we first use MegaBLAST to find its most similar sequence from the dataset. These two aligned sequences form a cluster, for which an obfuscated sequence was generated via a DNA generalization lattice scheme. These procedures are iteratively performed until all of the sequences in the dataset are clustered and their obfuscated sequences are generated. Experimental results on two benchmark datasets demonstrate that under the same degree of anonymity, IterMegaBLAST significantly outperforms existing state-of-the-art approaches in terms of both utility accuracy and time complexity.
Tasks	Drug Discovery
Published	2017-08-08
URL	http://arxiv.org/abs/1708.02629v1
PDF	http://arxiv.org/pdf/1708.02629v1.pdf
PWC	https://paperswithcode.com/paper/protecting-genomic-privacy-by-a-sequence
Repo
Framework


Title	Multilingual Multi-modal Embeddings for Natural Language Processing
Authors	Iacer Calixto, Qun Liu, Nick Campbell
Abstract	We propose a novel discriminative model that learns embeddings from multilingual and multi-modal data, meaning that our model can take advantage of images and descriptions in multiple languages to improve embedding quality. To that end, we introduce a modification of a pairwise contrastive estimation optimisation function as our training objective. We evaluate our embeddings on an image-sentence ranking (ISR), a semantic textual similarity (STS), and a neural machine translation (NMT) task. We find that the additional multilingual signals lead to improvements on both the ISR and STS tasks, and the discriminative cost can also be used in re-ranking $n$-best lists produced by NMT models, yielding strong improvements.
Tasks	Machine Translation, Semantic Textual Similarity
Published	2017-02-03
URL	http://arxiv.org/abs/1702.01101v1
PDF	http://arxiv.org/pdf/1702.01101v1.pdf
PWC	https://paperswithcode.com/paper/multilingual-multi-modal-embeddings-for
Repo
Framework

Subspace Clustering using Ensembles of $K$-Subspaces


Title	Subspace Clustering using Ensembles of $K$-Subspaces
Authors	John Lipor, David Hong, Yan Shuo Tan, Laura Balzano
Abstract	Subspace clustering is the unsupervised grouping of points lying near a union of low-dimensional linear subspaces. Algorithms based directly on geometric properties of such data tend to either provide poor empirical performance, lack theoretical guarantees, or depend heavily on their initialization. We present a novel geometric approach to the subspace clustering problem that leverages ensembles of the K-subspaces (KSS) algorithm via the evidence accumulation clustering framework. Our algorithm, referred to as ensemble K-subspaces (EKSS), forms a co-association matrix whose (i,j)th entry is the number of times points i and j are clustered together by several runs of KSS with random initializations. We prove general recovery guarantees for any algorithm that forms an affinity matrix with entries close to a monotonic transformation of pairwise absolute inner products. We then show that a specific instance of EKSS results in an affinity matrix with entries of this form, and hence our proposed algorithm can provably recover subspaces under similar conditions to state-of-the-art algorithms. The finding is, to the best of our knowledge, the first recovery guarantee for evidence accumulation clustering and for KSS variants. We show on synthetic data that our method performs well in the traditionally challenging settings of subspaces with large intersection, subspaces with small principal angles, and noisy data. Finally, we evaluate our algorithm on six common benchmark datasets and show that unlike existing methods, EKSS achieves excellent empirical performance when there are both a small and large number of points per subspace.
Tasks
Published	2017-09-14
URL	http://arxiv.org/abs/1709.04744v2
PDF	http://arxiv.org/pdf/1709.04744v2.pdf
PWC	https://paperswithcode.com/paper/subspace-clustering-using-ensembles-of-k
Repo
Framework

Solving DCOPs with Distributed Large Neighborhood Search


Title	Solving DCOPs with Distributed Large Neighborhood Search
Authors	Ferdinando Fioretto, Agostino Dovier, Enrico Pontelli, William Yeoh, Roie Zivan
Abstract	The field of Distributed Constraint Optimization has gained momentum in recent years, thanks to its ability to address various applications related to multi-agent cooperation. Nevertheless, solving Distributed Constraint Optimization Problems (DCOPs) optimally is NP-hard. Therefore, in large-scale, complex applications, incomplete DCOP algorithms are necessary. Current incomplete DCOP algorithms suffer of one or more of the following limitations: they (a) find local minima without providing quality guarantees; (b) provide loose quality assessment; or (c) are unable to benefit from the structure of the problem, such as domain-dependent knowledge and hard constraints. Therefore, capitalizing on strategies from the centralized constraint solving community, we propose a Distributed Large Neighborhood Search (D-LNS) framework to solve DCOPs. The proposed framework (with its novel repair phase) provides guarantees on solution quality, refining upper and lower bounds during the iterative process, and can exploit domain-dependent structures. Our experimental results show that D-LNS outperforms other incomplete DCOP algorithms on both structured and unstructured problem instances.
Tasks
Published	2017-02-22
URL	http://arxiv.org/abs/1702.06915v2
PDF	http://arxiv.org/pdf/1702.06915v2.pdf
PWC	https://paperswithcode.com/paper/solving-dcops-with-distributed-large
Repo
Framework

Jeffrey’s prior sampling of deep sigmoidal networks


Title	Jeffrey’s prior sampling of deep sigmoidal networks
Authors	Lorien X. Hayden, Alexander A. Alemi, Paul H. Ginsparg, James P. Sethna
Abstract	Neural networks have been shown to have a remarkable ability to uncover low dimensional structure in data: the space of possible reconstructed images form a reduced model manifold in image space. We explore this idea directly by analyzing the manifold learned by Deep Belief Networks and Stacked Denoising Autoencoders using Monte Carlo sampling. The model manifold forms an only slightly elongated hyperball with actual reconstructed data appearing predominantly on the boundaries of the manifold. In connection with the results we present, we discuss problems of sampling high-dimensional manifolds as well as recent work [M. Transtrum, G. Hart, and P. Qiu, Submitted (2014)] discussing the relation between high dimensional geometry and model reduction.
Tasks	Denoising
Published	2017-05-25
URL	http://arxiv.org/abs/1705.10589v1
PDF	http://arxiv.org/pdf/1705.10589v1.pdf
PWC	https://paperswithcode.com/paper/jeffreys-prior-sampling-of-deep-sigmoidal
Repo
Framework