May 6, 2019

3261 words 16 mins read

Paper Group ANR 368

Community Structure in Industrial SAT Instances. Self-calibrating Neural Networks for Dimensionality Reduction. Toward Organic Computing Approach for Cybernetic Responsive Environment. Self-Supervised Video Representation Learning With Odd-One-Out Networks. A Semi-Markov Switching Linear Gaussian Model for Censored Physiological Data. Sort Story: S …

Community Structure in Industrial SAT Instances


Title	Community Structure in Industrial SAT Instances
Authors	Carlos Ansótegui, Maria Luisa Bonet, Jesús Giráldez-Cru, Jordi Levy, Laurent Simon
Abstract	Modern SAT solvers have experienced a remarkable progress on solving industrial instances. Most of the techniques have been developed after an intensive experimental process. It is believed that these techniques exploit the underlying structure of industrial instances. However, there are few works trying to exactly characterize the main features of this structure. The research community on complex networks has developed techniques of analysis and algorithms to study real-world graphs that can be used by the SAT community. Recently, there have been some attempts to analyze the structure of industrial SAT instances in terms of complex networks, with the aim of explaining the success of SAT solving techniques, and possibly improving them. In this paper, inspired by the results on complex networks, we study the community structure, or modularity, of industrial SAT instances. In a graph with clear community structure, or high modularity, we can find a partition of its nodes into communities such that most edges connect variables of the same community. In our analysis, we represent SAT instances as graphs, and we show that most application benchmarks are characterized by a high modularity. On the contrary, random SAT instances are closer to the classical Erd"os-R'enyi random graph model, where no structure can be observed. We also analyze how this structure evolves by the effects of the execution of a CDCL SAT solver. In particular, we use the community structure to detect that new clauses learned by the solver during the search contribute to destroy the original structure of the formula. This is, learned clauses tend to contain variables of distinct communities.
Tasks
Published	2016-06-10
URL	https://arxiv.org/abs/1606.03329v3
PDF	https://arxiv.org/pdf/1606.03329v3.pdf
PWC	https://paperswithcode.com/paper/community-structure-in-industrial-sat
Repo
Framework

Self-calibrating Neural Networks for Dimensionality Reduction


Title	Self-calibrating Neural Networks for Dimensionality Reduction
Authors	Yuansi Chen, Cengiz Pehlevan, Dmitri B. Chklovskii
Abstract	Recently, a novel family of biologically plausible online algorithms for reducing the dimensionality of streaming data has been derived from the similarity matching principle. In these algorithms, the number of output dimensions can be determined adaptively by thresholding the singular values of the input data matrix. However, setting such threshold requires knowing the magnitude of the desired singular values in advance. Here we propose online algorithms where the threshold is self-calibrating based on the singular values computed from the existing observations. To derive these algorithms from the similarity matching cost function we propose novel regularizers. As before, these online algorithms can be implemented by Hebbian/anti-Hebbian neural networks in which the learning rule depends on the chosen regularizer. We demonstrate both mathematically and via simulation the effectiveness of these online algorithms in various settings.
Tasks	Dimensionality Reduction
Published	2016-12-11
URL	http://arxiv.org/abs/1612.03480v1
PDF	http://arxiv.org/pdf/1612.03480v1.pdf
PWC	https://paperswithcode.com/paper/self-calibrating-neural-networks-for
Repo
Framework

Toward Organic Computing Approach for Cybernetic Responsive Environment


Title	Toward Organic Computing Approach for Cybernetic Responsive Environment
Authors	Duhart Clément, Bertelle Cyrille
Abstract	The developpment of the Internet of Things (IoT) concept revives Responsive Environments (RE) technologies. Nowadays, the idea of a permanent connection between physical and digital world is technologically possible. The capillar Internet relates to the Internet extension into daily appliances such as they become actors of Internet like any hu-man. The parallel development of Machine-to-Machine communications and Arti cial Intelligence (AI) technics start a new area of cybernetic. This paper presents an approach for Cybernetic Organism (Cyborg) for RE based on Organic Computing (OC). In such approach, each appli-ance is a part of an autonomic system in order to control a physical environment. The underlying idea is that such systems must have self-x properties in order to adapt their behavior to external disturbances with a high-degree of autonomy.
Tasks
Published	2016-01-07
URL	http://arxiv.org/abs/1601.01614v1
PDF	http://arxiv.org/pdf/1601.01614v1.pdf
PWC	https://paperswithcode.com/paper/toward-organic-computing-approach-for
Repo
Framework

Self-Supervised Video Representation Learning With Odd-One-Out Networks


Title	Self-Supervised Video Representation Learning With Odd-One-Out Networks
Authors	Basura Fernando, Hakan Bilen, Efstratios Gavves, Stephen Gould
Abstract	We propose a new self-supervised CNN pre-training technique based on a novel auxiliary task called “odd-one-out learning”. In this task, the machine is asked to identify the unrelated or odd element from a set of otherwise related elements. We apply this technique to self-supervised video representation learning where we sample subsequences from videos and ask the network to learn to predict the odd video subsequence. The odd video subsequence is sampled such that it has wrong temporal order of frames while the even ones have the correct temporal order. Therefore, to generate a odd-one-out question no manual annotation is required. Our learning machine is implemented as multi-stream convolutional neural network, which is learned end-to-end. Using odd-one-out networks, we learn temporal representations for videos that generalizes to other related tasks such as action recognition. On action classification, our method obtains 60.3% on the UCF101 dataset using only UCF101 data for training which is approximately 10% better than current state-of-the-art self-supervised learning methods. Similarly, on HMDB51 dataset we outperform self-supervised state-of-the art methods by 12.7% on action classification task.
Tasks	Action Classification, Representation Learning, Temporal Action Localization
Published	2016-11-21
URL	http://arxiv.org/abs/1611.06646v4
PDF	http://arxiv.org/pdf/1611.06646v4.pdf
PWC	https://paperswithcode.com/paper/self-supervised-video-representation-learning-1
Repo
Framework

A Semi-Markov Switching Linear Gaussian Model for Censored Physiological Data


Title	A Semi-Markov Switching Linear Gaussian Model for Censored Physiological Data
Authors	Ahmed M. Alaa, Jinsung Yoon, Scott Hu, Mihaela van der Schaar
Abstract	Critically ill patients in regular wards are vulnerable to unanticipated clinical dete- rioration which requires timely transfer to the intensive care unit (ICU). To allow for risk scoring and patient monitoring in such a setting, we develop a novel Semi- Markov Switching Linear Gaussian Model (SSLGM) for the inpatients’ physiol- ogy. The model captures the patients’ latent clinical states and their corresponding observable lab tests and vital signs. We present an efficient unsupervised learn- ing algorithm that capitalizes on the informatively censored data in the electronic health records (EHR) to learn the parameters of the SSLGM; the learned model is then used to assess the new inpatients’ risk for clinical deterioration in an online fashion, allowing for timely ICU admission. Experiments conducted on a het- erogeneous cohort of 6,094 patients admitted to a large academic medical center show that the proposed model significantly outperforms the currently deployed risk scores such as Rothman index, MEWS, SOFA and APACHE.
Tasks
Published	2016-11-16
URL	http://arxiv.org/abs/1611.05146v1
PDF	http://arxiv.org/pdf/1611.05146v1.pdf
PWC	https://paperswithcode.com/paper/a-semi-markov-switching-linear-gaussian-model
Repo
Framework

Sort Story: Sorting Jumbled Images and Captions into Stories


Title	Sort Story: Sorting Jumbled Images and Captions into Stories
Authors	Harsh Agrawal, Arjun Chandrasekaran, Dhruv Batra, Devi Parikh, Mohit Bansal
Abstract	Temporal common sense has applications in AI tasks such as QA, multi-document summarization, and human-AI communication. We propose the task of sequencing – given a jumbled set of aligned image-caption pairs that belong to a story, the task is to sort them such that the output sequence forms a coherent story. We present multiple approaches, via unary (position) and pairwise (order) predictions, and their ensemble-based combinations, achieving strong results on this task. We use both text-based and image-based features, which depict complementary improvements. Using qualitative examples, we demonstrate that our models have learnt interesting aspects of temporal common sense.
Tasks	Common Sense Reasoning, Document Summarization, Multi-Document Summarization
Published	2016-06-23
URL	http://arxiv.org/abs/1606.07493v5
PDF	http://arxiv.org/pdf/1606.07493v5.pdf
PWC	https://paperswithcode.com/paper/sort-story-sorting-jumbled-images-and
Repo
Framework

Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes


Title	Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
Authors	Jack W Rae, Jonathan J Hunt, Tim Harley, Ivo Danihelka, Andrew Senior, Greg Wayne, Alex Graves, Timothy P Lillicrap
Abstract	Neural networks augmented with external memory have the ability to learn algorithmic solutions to complex tasks. These models appear promising for applications such as language modeling and machine translation. However, they scale poorly in both space and time as the amount of memory grows — limiting their applicability to real-world domains. Here, we present an end-to-end differentiable memory access scheme, which we call Sparse Access Memory (SAM), that retains the representational power of the original approaches whilst training efficiently with very large memories. We show that SAM achieves asymptotic lower bounds in space and time complexity, and find that an implementation runs $1,!000\times$ faster and with $3,!000\times$ less physical memory than non-sparse models. SAM learns with comparable data efficiency to existing models on a range of synthetic tasks and one-shot Omniglot character recognition, and can scale to tasks requiring $100,!000$s of time steps and memories. As well, we show how our approach can be adapted for models that maintain temporal associations between memories, as with the recently introduced Differentiable Neural Computer.
Tasks	Language Modelling, Machine Translation, Omniglot, Question Answering
Published	2016-10-27
URL	http://arxiv.org/abs/1610.09027v1
PDF	http://arxiv.org/pdf/1610.09027v1.pdf
PWC	https://paperswithcode.com/paper/scaling-memory-augmented-neural-networks-with
Repo
Framework

A Taxonomy of Deep Convolutional Neural Nets for Computer Vision


Title	A Taxonomy of Deep Convolutional Neural Nets for Computer Vision
Authors	Suraj Srinivas, Ravi Kiran Sarvadevabhatla, Konda Reddy Mopuri, Nikita Prabhu, Srinivas S S Kruthiventi, R. Venkatesh Babu
Abstract	Traditional architectures for solving computer vision problems and the degree of success they enjoyed have been heavily reliant on hand-crafted features. However, of late, deep learning techniques have offered a compelling alternative – that of automatically learning problem-specific features. With this new paradigm, every problem in computer vision is now being re-examined from a deep learning perspective. Therefore, it has become important to understand what kind of deep networks are suitable for a given problem. Although general surveys of this fast-moving paradigm (i.e. deep-networks) exist, a survey specific to computer vision is missing. We specifically consider one form of deep networks widely used in computer vision - convolutional neural networks (CNNs). We start with “AlexNet” as our base CNN and then examine the broad variations proposed over time to suit different applications. We hope that our recipe-style survey will serve as a guide, particularly for novice practitioners intending to use deep-learning techniques for computer vision.
Tasks
Published	2016-01-25
URL	http://arxiv.org/abs/1601.06615v1
PDF	http://arxiv.org/pdf/1601.06615v1.pdf
PWC	https://paperswithcode.com/paper/a-taxonomy-of-deep-convolutional-neural-nets
Repo
Framework

InstanceCut: from Edges to Instances with MultiCut


Title	InstanceCut: from Edges to Instances with MultiCut
Authors	Alexander Kirillov, Evgeny Levinkov, Bjoern Andres, Bogdan Savchynskyy, Carsten Rother
Abstract	This work addresses the task of instance-aware semantic segmentation. Our key motivation is to design a simple method with a new modelling-paradigm, which therefore has a different trade-off between advantages and disadvantages compared to known approaches. Our approach, we term InstanceCut, represents the problem by two output modalities: (i) an instance-agnostic semantic segmentation and (ii) all instance-boundaries. The former is computed from a standard convolutional neural network for semantic segmentation, and the latter is derived from a new instance-aware edge detection model. To reason globally about the optimal partitioning of an image into instances, we combine these two modalities into a novel MultiCut formulation. We evaluate our approach on the challenging CityScapes dataset. Despite the conceptual simplicity of our approach, we achieve the best result among all published methods, and perform particularly well for rare object classes.
Tasks	Edge Detection, Semantic Segmentation
Published	2016-11-24
URL	http://arxiv.org/abs/1611.08272v1
PDF	http://arxiv.org/pdf/1611.08272v1.pdf
PWC	https://paperswithcode.com/paper/instancecut-from-edges-to-instances-with
Repo
Framework

SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoods


Title	SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoods
Authors	Marzieh Saeidi, Guillaume Bouchard, Maria Liakata, Sebastian Riedel
Abstract	In this paper, we introduce the task of targeted aspect-based sentiment analysis. The goal is to extract fine-grained information with respect to entities mentioned in user comments. This work extends both aspect-based sentiment analysis that assumes a single entity per document and targeted sentiment analysis that assumes a single sentiment towards a target entity. In particular, we identify the sentiment towards each aspect of one or more entities. As a testbed for this task, we introduce the SentiHood dataset, extracted from a question answering (QA) platform where urban neighbourhoods are discussed by users. In this context units of text often mention several aspects of one or more neighbourhoods. This is the first time that a generic social media platform in this case a QA platform, is used for fine-grained opinion mining. Text coming from QA platforms is far less constrained compared to text from review specific platforms which current datasets are based on. We develop several strong baselines, relying on logistic regression and state-of-the-art recurrent neural networks.
Tasks	Aspect-Based Sentiment Analysis, Opinion Mining, Question Answering, Sentiment Analysis
Published	2016-10-12
URL	http://arxiv.org/abs/1610.03771v1
PDF	http://arxiv.org/pdf/1610.03771v1.pdf
PWC	https://paperswithcode.com/paper/sentihood-targeted-aspect-based-sentiment
Repo
Framework

Variational perturbation and extended Plefka approaches to dynamics on random networks: the case of the kinetic Ising model


Title	Variational perturbation and extended Plefka approaches to dynamics on random networks: the case of the kinetic Ising model
Authors	Ludovica Bachschmid-Romano, Claudia Battistin, Manfred Opper, Yasser Roudi
Abstract	We describe and analyze some novel approaches for studying the dynamics of Ising spin glass models. We first briefly consider the variational approach based on minimizing the Kullback-Leibler divergence between independent trajectories and the real ones and note that this approach only coincides with the mean field equations from the saddle point approximation to the generating functional when the dynamics is defined through a logistic link function, which is the case for the kinetic Ising model with parallel update. We then spend the rest of the paper developing two ways of going beyond the saddle point approximation to the generating functional. In the first one, we develop a variational perturbative approximation to the generating functional by expanding the action around a quadratic function of the local fields and conjugate local fields whose parameters are optimized. We derive analytical expressions for the optimal parameters and show that when the optimization is suitably restricted, we recover the mean field equations that are exact for the fully asymmetric random couplings (M'ezard and Sakellariou, 2011). However, without this restriction the results are different. We also describe an extended Plefka expansion in which in addition to the magnetization, we also fix the correlation and response functions. Finally, we numerically study the performance of these approximations for Sherrington-Kirkpatrick type couplings for various coupling strengths, degrees of coupling symmetry and external fields. We show that the dynamical equations derived from the extended Plefka expansion outperform the others in all regimes, although it is computationally more demanding. The unconstrained variational approach does not perform well in the small coupling regime, while it approaches dynamical TAP equations of (Roudi and Hertz, 2011) for strong couplings.
Tasks
Published	2016-07-28
URL	http://arxiv.org/abs/1607.08379v1
PDF	http://arxiv.org/pdf/1607.08379v1.pdf
PWC	https://paperswithcode.com/paper/variational-perturbation-and-extended-plefka
Repo
Framework

Real-time interactive sequence generation and control with Recurrent Neural Network ensembles


Title	Real-time interactive sequence generation and control with Recurrent Neural Network ensembles
Authors	Memo Akten, Mick Grierson
Abstract	Recurrent Neural Networks (RNN), particularly Long Short Term Memory (LSTM) RNNs, are a popular and very successful method for learning and generating sequences. However, current generative RNN techniques do not allow real-time interactive control of the sequence generation process, thus aren’t well suited for live creative expression. We propose a method of real-time continuous control and ‘steering’ of sequence generation using an ensemble of RNNs and dynamically altering the mixture weights of the models. We demonstrate the method using character based LSTM networks and a gestural interface allowing users to ‘conduct’ the generation of text.
Tasks	Continuous Control
Published	2016-12-14
URL	http://arxiv.org/abs/1612.04687v2
PDF	http://arxiv.org/pdf/1612.04687v2.pdf
PWC	https://paperswithcode.com/paper/real-time-interactive-sequence-generation-and
Repo
Framework

An Online Structural Plasticity Rule for Generating Better Reservoirs


Title	An Online Structural Plasticity Rule for Generating Better Reservoirs
Authors	Subhrajit Roy, Arindam Basu
Abstract	In this article, a novel neuro-inspired low-resolution online unsupervised learning rule is proposed to train the reservoir or liquid of Liquid State Machine. The liquid is a sparsely interconnected huge recurrent network of spiking neurons. The proposed learning rule is inspired from structural plasticity and trains the liquid through formation and elimination of synaptic connections. Hence, the learning involves rewiring of the reservoir connections similar to structural plasticity observed in biological neural networks. The network connections can be stored as a connection matrix and updated in memory by using Address Event Representation (AER) protocols which are generally employed in neuromorphic systems. On investigating the ‘pairwise separation property’ we find that trained liquids provide 1.36 $\pm$ 0.18 times more inter-class separation while retaining similar intra-class separation as compared to random liquids. Moreover, analysis of the ‘linear separation property’ reveals that trained liquids are 2.05 $\pm$ 0.27 times better than random liquids. Furthermore, we show that our liquids are able to retain the ‘generalization’ ability and ‘generality’ of random liquids. A memory analysis shows that trained liquids have 83.67 $\pm$ 5.79 ms longer fading memory than random liquids which have shown 92.8 $\pm$ 5.03 ms fading memory for a particular type of spike train inputs. We also throw some light on the dynamics of the evolution of recurrent connections within the liquid. Moreover, compared to ‘Separation Driven Synaptic Modification’ - a recently proposed algorithm for iteratively refining reservoirs, our learning rule provides 9.30%, 15.21% and 12.52% more liquid separations and 2.8%, 9.1% and 7.9% better classification accuracies for four, eight and twelve class pattern recognition tasks respectively.
Tasks
Published	2016-04-19
URL	http://arxiv.org/abs/1604.05459v1
PDF	http://arxiv.org/pdf/1604.05459v1.pdf
PWC	https://paperswithcode.com/paper/an-online-structural-plasticity-rule-for
Repo
Framework

Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling


Title	Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling
Authors	Zhe Gan, Chunyuan Li, Changyou Chen, Yunchen Pu, Qinliang Su, Lawrence Carin
Abstract	Recurrent neural networks (RNNs) have shown promising performance for language modeling. However, traditional training of RNNs using back-propagation through time often suffers from overfitting. One reason for this is that stochastic optimization (used for large training sets) does not provide good estimates of model uncertainty. This paper leverages recent advances in stochastic gradient Markov Chain Monte Carlo (also appropriate for large training sets) to learn weight uncertainty in RNNs. It yields a principled Bayesian learning algorithm, adding gradient noise during training (enhancing exploration of the model-parameter space) and model averaging when testing. Extensive experiments on various RNN models and across a broad range of applications demonstrate the superiority of the proposed approach over stochastic optimization.
Tasks	Language Modelling, Stochastic Optimization
Published	2016-11-23
URL	http://arxiv.org/abs/1611.08034v2
PDF	http://arxiv.org/pdf/1611.08034v2.pdf
PWC	https://paperswithcode.com/paper/scalable-bayesian-learning-of-recurrent
Repo
Framework

Scene Text Detection via Holistic, Multi-Channel Prediction


Title	Scene Text Detection via Holistic, Multi-Channel Prediction
Authors	Cong Yao, Xiang Bai, Nong Sang, Xinyu Zhou, Shuchang Zhou, Zhimin Cao
Abstract	Recently, scene text detection has become an active research topic in computer vision and document analysis, because of its great importance and significant challenge. However, vast majority of the existing methods detect text within local regions, typically through extracting character, word or line level candidates followed by candidate aggregation and false positive elimination, which potentially exclude the effect of wide-scope and long-range contextual cues in the scene. To take full advantage of the rich information available in the whole natural image, we propose to localize text in a holistic manner, by casting scene text detection as a semantic segmentation problem. The proposed algorithm directly runs on full images and produces global, pixel-wise prediction maps, in which detections are subsequently formed. To better make use of the properties of text, three types of information regarding text region, individual characters and their relationship are estimated, with a single Fully Convolutional Network (FCN) model. With such predictions of text properties, the proposed algorithm can simultaneously handle horizontal, multi-oriented and curved text in real-world natural images. The experiments on standard benchmarks, including ICDAR 2013, ICDAR 2015 and MSRA-TD500, demonstrate that the proposed algorithm substantially outperforms previous state-of-the-art approaches. Moreover, we report the first baseline result on the recently-released, large-scale dataset COCO-Text.
Tasks	Scene Text Detection, Semantic Segmentation
Published	2016-06-29
URL	http://arxiv.org/abs/1606.09002v2
PDF	http://arxiv.org/pdf/1606.09002v2.pdf
PWC	https://paperswithcode.com/paper/scene-text-detection-via-holistic-multi
Repo
Framework