May 7, 2019

3343 words 16 mins read

Paper Group ANR 3

Fast and Efficient Asynchronous Neural Computation with Adapting Spiking Neural Networks. Stream Reasoning-Based Control of Caching Strategies in CCN Routers. First Efficient Convergence for Streaming k-PCA: a Global, Gap-Free, and Near-Optimal Rate. Federated Learning: Strategies for Improving Communication Efficiency. Online Multi-Target Tracking …

Fast and Efficient Asynchronous Neural Computation with Adapting Spiking Neural Networks


Title	Fast and Efficient Asynchronous Neural Computation with Adapting Spiking Neural Networks
Authors	Davide Zambrano, Sander M. Bohte
Abstract	Biological neurons communicate with a sparing exchange of pulses - spikes. It is an open question how real spiking neurons produce the kind of powerful neural computation that is possible with deep artificial neural networks, using only so very few spikes to communicate. Building on recent insights in neuroscience, we present an Adapting Spiking Neural Network (ASNN) based on adaptive spiking neurons. These spiking neurons efficiently encode information in spike-trains using a form of Asynchronous Pulsed Sigma-Delta coding while homeostatically optimizing their firing rate. In the proposed paradigm of spiking neuron computation, neural adaptation is tightly coupled to synaptic plasticity, to ensure that downstream neurons can correctly decode upstream spiking neurons. We show that this type of network is inherently able to carry out asynchronous and event-driven neural computation, while performing identical to corresponding artificial neural networks (ANNs). In particular, we show that these adaptive spiking neurons can be drop in replacements for ReLU neurons in standard feedforward ANNs comprised of such units. We demonstrate that this can also be successfully applied to a ReLU based deep convolutional neural network for classifying the MNIST dataset. The ASNN thus outperforms current Spiking Neural Networks (SNNs) implementations, while responding (up to) an order of magnitude faster and using an order of magnitude fewer spikes. Additionally, in a streaming setting where frames are continuously classified, we show that the ASNN requires substantially fewer network updates as compared to the corresponding ANN.
Tasks
Published	2016-09-07
URL	http://arxiv.org/abs/1609.02053v1
PDF	http://arxiv.org/pdf/1609.02053v1.pdf
PWC	https://paperswithcode.com/paper/fast-and-efficient-asynchronous-neural
Repo
Framework

Stream Reasoning-Based Control of Caching Strategies in CCN Routers


Title	Stream Reasoning-Based Control of Caching Strategies in CCN Routers
Authors	Harald Beck, Bruno Bierbaumer, Minh Dao-Tran, Thomas Eiter, Hermann Hellwagner, Konstantin Schekotihin
Abstract	Content-Centric Networking (CCN) research addresses the mismatch between the modern usage of the Internet and its outdated architecture. Importantly, CCN routers may locally cache frequently requested content in order to speed up delivery to end users. Thus, the issue of caching strategies arises, i.e., which content shall be stored and when it should be replaced. In this work, we employ novel techniques towards intelligent administration of CCN routers that autonomously switch between existing strategies in response to changing content request patterns. In particular, we present a router architecture for CCN networks that is controlled by rule-based stream reasoning, following the recent formal framework LARS which extends Answer Set Programming for streams. The obtained possibility for flexible router configuration at runtime allows for faster experimentation and may thus help to advance the further development of CCN. Moreover, the empirical evaluation of our feasibility study shows that the resulting caching agent may give significant performance gains.
Tasks
Published	2016-10-13
URL	http://arxiv.org/abs/1610.04005v1
PDF	http://arxiv.org/pdf/1610.04005v1.pdf
PWC	https://paperswithcode.com/paper/stream-reasoning-based-control-of-caching
Repo
Framework

First Efficient Convergence for Streaming k-PCA: a Global, Gap-Free, and Near-Optimal Rate


Title	First Efficient Convergence for Streaming k-PCA: a Global, Gap-Free, and Near-Optimal Rate
Authors	Zeyuan Allen-Zhu, Yuanzhi Li
Abstract	We study streaming principal component analysis (PCA), that is to find, in $O(dk)$ space, the top $k$ eigenvectors of a $d\times d$ hidden matrix $\bf \Sigma$ with online vectors drawn from covariance matrix $\bf \Sigma$. We provide $\textit{global}$ convergence for Oja’s algorithm which is popularly used in practice but lacks theoretical understanding for $k>1$. We also provide a modified variant $\mathsf{Oja}^{++}$ that runs $\textit{even faster}$ than Oja’s. Our results match the information theoretic lower bound in terms of dependency on error, on eigengap, on rank $k$, and on dimension $d$, up to poly-log factors. In addition, our convergence rate can be made gap-free, that is proportional to the approximation error and independent of the eigengap. In contrast, for general rank $k$, before our work (1) it was open to design any algorithm with efficient global convergence rate; and (2) it was open to design any algorithm with (even local) gap-free convergence rate in $O(dk)$ space.
Tasks
Published	2016-07-26
URL	http://arxiv.org/abs/1607.07837v4
PDF	http://arxiv.org/pdf/1607.07837v4.pdf
PWC	https://paperswithcode.com/paper/first-efficient-convergence-for-streaming-k
Repo
Framework

Federated Learning: Strategies for Improving Communication Efficiency


Title	Federated Learning: Strategies for Improving Communication Efficiency
Authors	Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, Dave Bacon
Abstract	Federated Learning is a machine learning setting where the goal is to train a high-quality centralized model while training data remains distributed over a large number of clients each with unreliable and relatively slow network connections. We consider learning algorithms for this setting where on each round, each client independently computes an update to the current model based on its local data, and communicates this update to a central server, where the client-side updates are aggregated to compute a new global model. The typical clients in this setting are mobile phones, and communication efficiency is of the utmost importance. In this paper, we propose two ways to reduce the uplink communication costs: structured updates, where we directly learn an update from a restricted space parametrized using a smaller number of variables, e.g. either low-rank or a random mask; and sketched updates, where we learn a full model update and then compress it using a combination of quantization, random rotations, and subsampling before sending it to the server. Experiments on both convolutional and recurrent networks show that the proposed methods can reduce the communication cost by two orders of magnitude.
Tasks	Quantization
Published	2016-10-18
URL	http://arxiv.org/abs/1610.05492v2
PDF	http://arxiv.org/pdf/1610.05492v2.pdf
PWC	https://paperswithcode.com/paper/federated-learning-strategies-for-improving
Repo
Framework

Online Multi-Target Tracking Using Recurrent Neural Networks


Title	Online Multi-Target Tracking Using Recurrent Neural Networks
Authors	Anton Milan, Seyed Hamid Rezatofighi, Anthony Dick, Ian Reid, Konrad Schindler
Abstract	We present a novel approach to online multi-target tracking based on recurrent neural networks (RNNs). Tracking multiple objects in real-world scenes involves many challenges, including a) an a-priori unknown and time-varying number of targets, b) a continuous state estimation of all present targets, and c) a discrete combinatorial problem of data association. Most previous methods involve complex models that require tedious tuning of parameters. Here, we propose for the first time, an end-to-end learning approach for online multi-target tracking. Existing deep learning methods are not designed for the above challenges and cannot be trivially applied to the task. Our solution addresses all of the above points in a principled way. Experiments on both synthetic and real data show promising results obtained at ~300 Hz on a standard CPU, and pave the way towards future research in this direction.
Tasks
Published	2016-04-13
URL	http://arxiv.org/abs/1604.03635v2
PDF	http://arxiv.org/pdf/1604.03635v2.pdf
PWC	https://paperswithcode.com/paper/online-multi-target-tracking-using-recurrent
Repo
Framework

Learning to Generate Genotypes with Neural Networks


Title	Learning to Generate Genotypes with Neural Networks
Authors	Alexander W. Churchill, Siddharth Sigtia, Chrisantha Fernando
Abstract	Neural networks and evolutionary computation have a rich intertwined history. They most commonly appear together when an evolutionary algorithm optimises the parameters and topology of a neural network for reinforcement learning problems, or when a neural network is applied as a surrogate fitness function to aid the evolutionary optimisation of expensive fitness functions. In this paper we take a different approach, asking the question of whether a neural network can be used to provide a mutation distribution for an evolutionary algorithm, and what advantages this approach may offer? Two modern neural network models are investigated, a Denoising Autoencoder modified to produce stochastic outputs and the Neural Autoregressive Distribution Estimator. Results show that the neural network approach to learning genotypes is able to solve many difficult discrete problems, such as MaxSat and HIFF, and regularly outperforms other evolutionary techniques.
Tasks	Denoising
Published	2016-04-14
URL	http://arxiv.org/abs/1604.04153v1
PDF	http://arxiv.org/pdf/1604.04153v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-generate-genotypes-with-neural
Repo
Framework

Towards a Deep Learning Framework for Unconstrained Face Detection


Title	Towards a Deep Learning Framework for Unconstrained Face Detection
Authors	Yutong Zheng, Chenchen Zhu, Khoa Luu, Chandrasekhar Bhagavatula, T. Hoang Ngan Le, Marios Savvides
Abstract	Robust face detection is one of the most important pre-processing steps to support facial expression analysis, facial landmarking, face recognition, pose estimation, building of 3D facial models, etc. Although this topic has been intensely studied for decades, it is still challenging due to numerous variants of face images in real-world scenarios. In this paper, we present a novel approach named Multiple Scale Faster Region-based Convolutional Neural Network (MS-FRCNN) to robustly detect human facial regions from images collected under various challenging conditions, e.g. large occlusions, extremely low resolutions, facial expressions, strong illumination variations, etc. The proposed approach is benchmarked on two challenging face detection databases, i.e. the Wider Face database and the Face Detection Dataset and Benchmark (FDDB), and compared against recent other face detection methods, e.g. Two-stage CNN, Multi-scale Cascade CNN, Faceness, Aggregate Chanel Features, HeadHunter, Multi-view Face Detection, Cascade CNN, etc. The experimental results show that our proposed approach consistently achieves highly competitive results with the state-of-the-art performance against other recent face detection methods.
Tasks	Face Detection, Face Recognition, Pose Estimation, Robust Face Recognition
Published	2016-12-16
URL	http://arxiv.org/abs/1612.05322v2
PDF	http://arxiv.org/pdf/1612.05322v2.pdf
PWC	https://paperswithcode.com/paper/towards-a-deep-learning-framework-for
Repo
Framework

Learning Adversary-Resistant Deep Neural Networks


Title	Learning Adversary-Resistant Deep Neural Networks
Authors	Qinglong Wang, Wenbo Guo, Kaixuan Zhang, Alexander G. Ororbia II, Xinyu Xing, Xue Liu, C. Lee Giles
Abstract	Deep neural networks (DNNs) have proven to be quite effective in a vast array of machine learning tasks, with recent examples in cyber security and autonomous vehicles. Despite the superior performance of DNNs in these applications, it has been recently shown that these models are susceptible to a particular type of attack that exploits a fundamental flaw in their design. This attack consists of generating particular synthetic examples referred to as adversarial samples. These samples are constructed by slightly manipulating real data-points in order to “fool” the original DNN model, forcing it to mis-classify previously correctly classified samples with high confidence. Addressing this flaw in the model is essential if DNNs are to be used in critical applications such as those in cyber security. Previous work has provided various learning algorithms to enhance the robustness of DNN models, and they all fall into the tactic of “security through obscurity”. This means security can be guaranteed only if one can obscure the learning algorithms from adversaries. Once the learning technique is disclosed, DNNs protected by these defense mechanisms are still susceptible to adversarial samples. In this work, we investigate this issue shared across previous research work and propose a generic approach to escalate a DNN’s resistance to adversarial samples. More specifically, our approach integrates a data transformation module with a DNN, making it robust even if we reveal the underlying learning algorithm. To demonstrate the generality of our proposed approach and its potential for handling cyber security applications, we evaluate our method and several other existing solutions on datasets publicly available. Our results indicate that our approach typically provides superior classification performance and resistance in comparison with state-of-art solutions.
Tasks	Autonomous Vehicles
Published	2016-12-05
URL	http://arxiv.org/abs/1612.01401v2
PDF	http://arxiv.org/pdf/1612.01401v2.pdf
PWC	https://paperswithcode.com/paper/learning-adversary-resistant-deep-neural
Repo
Framework

Efficient Learning with a Family of Nonconvex Regularizers by Redistributing Nonconvexity


Title	Efficient Learning with a Family of Nonconvex Regularizers by Redistributing Nonconvexity
Authors	Quanming Yao, James. T Kwok
Abstract	The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a large class of nonconvex regularizers, we propose to move the nonconvexity from the regularizer to the loss. The nonconvex regularizer is then transformed to a familiar convex regularizer, while the resultant loss function can still be guaranteed to be smooth. Learning with the convexified regularizer can be performed by existing efficient algorithms originally designed for convex regularizers (such as the proximal algorithm, Frank-Wolfe algorithm, alternating direction method of multipliers and stochastic gradient descent). Extensions are made when the convexified regularizer does not have closed-form proximal step, and when the loss function is nonconvex, nonsmooth. Extensive experiments on a variety of machine learning application scenarios show that optimizing the transformed problem is much faster than running the state-of-the-art on the original problem.
Tasks
Published	2016-06-13
URL	http://arxiv.org/abs/1606.03841v3
PDF	http://arxiv.org/pdf/1606.03841v3.pdf
PWC	https://paperswithcode.com/paper/efficient-learning-with-a-family-of-nonconvex
Repo
Framework

Image Co-localization by Mimicking a Good Detector’s Confidence Score Distribution


Title	Image Co-localization by Mimicking a Good Detector’s Confidence Score Distribution
Authors	Yao Li, Linqiao Liu, Chunhua Shen, Anton van den Hengel
Abstract	Given a set of images containing objects from the same category, the task of image co-localization is to identify and localize each instance. This paper shows that this problem can be solved by a simple but intriguing idea, that is, a common object detector can be learnt by making its detection confidence scores distributed like those of a strongly supervised detector. More specifically, we observe that given a set of object proposals extracted from an image that contains the object of interest, an accurate strongly supervised object detector should give high scores to only a small minority of proposals, and low scores to most of them. Thus, we devise an entropy-based objective function to enforce the above property when learning the common object detector. Once the detector is learnt, we resort to a segmentation approach to refine the localization. We show that despite its simplicity, our approach outperforms state-of-the-art methods.
Tasks
Published	2016-03-15
URL	http://arxiv.org/abs/1603.04619v2
PDF	http://arxiv.org/pdf/1603.04619v2.pdf
PWC	https://paperswithcode.com/paper/image-co-localization-by-mimicking-a-good
Repo
Framework


Title	Deep learning is effective for the classification of OCT images of normal versus Age-related Macular Degeneration
Authors	Cecilia S. Lee, Doug M. Baughman, Aaron Y. Lee
Abstract	Objective: The advent of Electronic Medical Records (EMR) with large electronic imaging databases along with advances in deep neural networks with machine learning has provided a unique opportunity to achieve milestones in automated image analysis. Optical coherence tomography (OCT) is the most commonly obtained imaging modality in ophthalmology and represents a dense and rich dataset when combined with labels derived from the EMR. We sought to determine if deep learning could be utilized to distinguish normal OCT images from images from patients with Age-related Macular Degeneration (AMD). Methods: Automated extraction of an OCT imaging database was performed and linked to clinical endpoints from the EMR. OCT macula scans were obtained by Heidelberg Spectralis, and each OCT scan was linked to EMR clinical endpoints extracted from EPIC. The central 11 images were selected from each OCT scan of two cohorts of patients: normal and AMD. Cross-validation was performed using a random subset of patients. Area under receiver operator curves (auROC) were constructed at an independent image level, macular OCT level, and patient level. Results: Of an extraction of 2.6 million OCT images linked to clinical datapoints from the EMR, 52,690 normal and 48,312 AMD macular OCT images were selected. A deep neural network was trained to categorize images as either normal or AMD. At the image level, we achieved an auROC of 92.78% with an accuracy of 87.63%. At the macula level, we achieved an auROC of 93.83% with an accuracy of 88.98%. At a patient level, we achieved an auROC of 97.45% with an accuracy of 93.45%. Peak sensitivity and specificity with optimal cutoffs were 92.64% and 93.69% respectively. Conclusions: Deep learning techniques are effective for classifying OCT images. These findings have important implications in utilizing OCT in automated screening and computer aided diagnosis tools.
Tasks
Published	2016-12-15
URL	http://arxiv.org/abs/1612.04891v1
PDF	http://arxiv.org/pdf/1612.04891v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-is-effective-for-the
Repo
Framework

A theory of interpretive clustering in free recall


Title	A theory of interpretive clustering in free recall
Authors	Francesco Fumarola
Abstract	A stochastic model of short-term verbal memory is proposed, in which the psychological state of the subject is encoded as the instantaneous position of a particle diffusing over a semantic graph with a probabilistic structure. The model is particularly suitable for studying the dependence of free-recall observables on semantic properties of the words to be recalled. Besides predicting some well-known experimental features (contiguity effect, forward asymmetry, word-length effect), a novel prediction is obtained on the relationship between the contiguity effect and the syllabic length of words; shorter words, by way of their wider semantic range, are predicted to be characterized by stronger forward contiguity. A fresh analysis of archival data allows to confirm this prediction.
Tasks
Published	2016-11-27
URL	http://arxiv.org/abs/1611.08928v2
PDF	http://arxiv.org/pdf/1611.08928v2.pdf
PWC	https://paperswithcode.com/paper/a-theory-of-interpretive-clustering-in-free
Repo
Framework

Collision-based Testers are Optimal for Uniformity and Closeness


Title	Collision-based Testers are Optimal for Uniformity and Closeness
Authors	Ilias Diakonikolas, Themis Gouleakis, John Peebles, Eric Price
Abstract	We study the fundamental problems of (i) uniformity testing of a discrete distribution, and (ii) closeness testing between two discrete distributions with bounded $\ell_2$-norm. These problems have been extensively studied in distribution testing and sample-optimal estimators are known for them~\cite{Paninski:08, CDVV14, VV14, DKN:15}. In this work, we show that the original collision-based testers proposed for these problems ~\cite{GRdist:00, BFR+:00} are sample-optimal, up to constant factors. Previous analyses showed sample complexity upper bounds for these testers that are optimal as a function of the domain size $n$, but suboptimal by polynomial factors in the error parameter $\epsilon$. Our main contribution is a new tight analysis establishing that these collision-based testers are information-theoretically optimal, up to constant factors, both in the dependence on $n$ and in the dependence on $\epsilon$.
Tasks
Published	2016-11-11
URL	http://arxiv.org/abs/1611.03579v1
PDF	http://arxiv.org/pdf/1611.03579v1.pdf
PWC	https://paperswithcode.com/paper/collision-based-testers-are-optimal-for
Repo
Framework

Local Binary Pattern for Word Spotting in Handwritten Historical Document


Title	Local Binary Pattern for Word Spotting in Handwritten Historical Document
Authors	Sounak Dey, Anguelos Nicolaou, Josep Llados, Umapada Pal
Abstract	Digital libraries store images which can be highly degraded and to index this kind of images we resort to word spot- ting as our information retrieval system. Information retrieval for handwritten document images is more challenging due to the difficulties in complex layout analysis, large variations of writing styles, and degradation or low quality of historical manuscripts. This paper presents a simple innovative learning-free method for word spotting from large scale historical documents combining Local Binary Pattern (LBP) and spatial sampling. This method offers three advantages: firstly, it operates in completely learning free paradigm which is very different from unsupervised learning methods, secondly, the computational time is significantly low because of the LBP features which are very fast to compute, and thirdly, the method can be used in scenarios where annotations are not available. Finally we compare the results of our proposed retrieval method with the other methods in the literature.
Tasks	Information Retrieval
Published	2016-04-20
URL	http://arxiv.org/abs/1604.05907v2
PDF	http://arxiv.org/pdf/1604.05907v2.pdf
PWC	https://paperswithcode.com/paper/local-binary-pattern-for-word-spotting-in
Repo
Framework

Proceedings First Workshop on Causal Reasoning for Embedded and safety-critical Systems Technologies


Title	Proceedings First Workshop on Causal Reasoning for Embedded and safety-critical Systems Technologies
Authors	Gregor Gössler, Oleg Sokolsky
Abstract	Formal approaches for automated causality analysis, fault localization, explanation of events, accountability and blaming have been proposed independently by several communities — in particular, AI, concurrency, model-based diagnosis, formal methods. Work on these topics has significantly gained speed during the last years. The goals of CREST are to bring together and foster exchange between researchers from the different communities, and to present and discuss recent advances and new ideas in the field. The workshop program consisted of a set of invited and contributed presentations that illustrate different techniques for, and applications of, causality analysis and fault localization. The program was anchored by two keynote talks. The keynote by Hana Chockler (King’s College) provided a broad perspective on the application of causal reasoning based on Halpern and Pearl’s definitions of actual causality to a variety of application domains ranging from formal verification to legal reasoning. The keynote by Chao Wang (Virginia Tech) concentrated on constraint-based analysis techniques for debugging and verifying concurrent programs. Workshop papers deal with compositional causality analysis and a wide spectrum of application for causal reasoning, such as debugging of probabilistic models, accountability and responsibility, hazard analysis in practice based on Lewis’ counterfactuals, and fault localization and repair.
Tasks
Published	2016-08-26
URL	http://arxiv.org/abs/1608.07398v1
PDF	http://arxiv.org/pdf/1608.07398v1.pdf
PWC	https://paperswithcode.com/paper/proceedings-first-workshop-on-causal
Repo
Framework