May 7, 2019

3158 words 15 mins read

Paper Group ANR 54

CNNLab: a Novel Parallel Framework for Neural Networks using GPU and FPGA-a Practical Study with Trade-off Analysis. Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification. Scalable and Sustainable Deep Learning via Randomized Hashing. Semidefinite Programs for Exact Recovery of a Hidden Community. Stacking machin …

CNNLab: a Novel Parallel Framework for Neural Networks using GPU and FPGA-a Practical Study with Trade-off Analysis


Title	CNNLab: a Novel Parallel Framework for Neural Networks using GPU and FPGA-a Practical Study with Trade-off Analysis
Authors	Maohua Zhu, Liu Liu, Chao Wang, Yuan Xie
Abstract	Designing and implementing efficient, provably correct parallel neural network processing is challenging. Existing high-level parallel abstractions like MapReduce are insufficiently expressive while low-level tools like MPI and Pthreads leave ML experts repeatedly solving the same design challenges. However, the diversity and large-scale data size have posed a significant challenge to construct a flexible and high-performance implementation of deep learning neural networks. To improve the performance and maintain the scalability, we present CNNLab, a novel deep learning framework using GPU and FPGA-based accelerators. CNNLab provides a uniform programming model to users so that the hardware implementation and the scheduling are invisible to the programmers. At runtime, CNNLab leverages the trade-offs between GPU and FPGA before offloading the tasks to the accelerators. Experimental results on the state-of-the-art Nvidia K40 GPU and Altera DE5 FPGA board demonstrate that the CNNLab can provide a universal framework with efficient support for diverse applications without increasing the burden of the programmers. Moreover, we analyze the detailed quantitative performance, throughput, power, energy, and performance density for both approaches. Experimental results leverage the trade-offs between GPU and FPGA and provide useful practical experiences for the deep learning research community.
Tasks
Published	2016-06-20
URL	http://arxiv.org/abs/1606.06234v1
PDF	http://arxiv.org/pdf/1606.06234v1.pdf
PWC	https://paperswithcode.com/paper/cnnlab-a-novel-parallel-framework-for-neural
Repo
Framework

Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification


Title	Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification
Authors	Jiacheng Xu, Danlu Chen, Xipeng Qiu, Xuangjing Huang
Abstract	Recently, neural networks have achieved great success on sentiment classification due to their ability to alleviate feature engineering. However, one of the remaining challenges is to model long texts in document-level sentiment classification under a recurrent architecture because of the deficiency of the memory unit. To address this problem, we present a Cached Long Short-Term Memory neural networks (CLSTM) to capture the overall semantic information in long texts. CLSTM introduces a cache mechanism, which divides memory into several groups with different forgetting rates and thus enables the network to keep sentiment information better within a recurrent unit. The proposed CLSTM outperforms the state-of-the-art models on three publicly available document-level sentiment analysis datasets.
Tasks	Feature Engineering, Sentiment Analysis
Published	2016-10-17
URL	http://arxiv.org/abs/1610.04989v1
PDF	http://arxiv.org/pdf/1610.04989v1.pdf
PWC	https://paperswithcode.com/paper/cached-long-short-term-memory-neural-networks
Repo
Framework

Scalable and Sustainable Deep Learning via Randomized Hashing


Title	Scalable and Sustainable Deep Learning via Randomized Hashing
Authors	Ryan Spring, Anshumali Shrivastava
Abstract	Current deep learning architectures are growing larger in order to learn from complex datasets. These architectures require giant matrix multiplication operations to train millions of parameters. Conversely, there is another growing trend to bring deep learning to low-power, embedded devices. The matrix operations, associated with both training and testing of deep networks, are very expensive from a computational and energy standpoint. We present a novel hashing based technique to drastically reduce the amount of computation needed to train and test deep networks. Our approach combines recent ideas from adaptive dropouts and randomized hashing for maximum inner product search to select the nodes with the highest activation efficiently. Our new algorithm for deep learning reduces the overall computational cost of forward and back-propagation by operating on significantly fewer (sparse) nodes. As a consequence, our algorithm uses only 5% of the total multiplications, while keeping on average within 1% of the accuracy of the original model. A unique property of the proposed hashing based back-propagation is that the updates are always sparse. Due to the sparse gradient updates, our algorithm is ideally suited for asynchronous and parallel training leading to near linear speedup with increasing number of cores. We demonstrate the scalability and sustainability (energy efficiency) of our proposed algorithm via rigorous experimental evaluations on several real datasets.
Tasks
Published	2016-02-26
URL	http://arxiv.org/abs/1602.08194v2
PDF	http://arxiv.org/pdf/1602.08194v2.pdf
PWC	https://paperswithcode.com/paper/scalable-and-sustainable-deep-learning-via
Repo
Framework

Semidefinite Programs for Exact Recovery of a Hidden Community


Title	Semidefinite Programs for Exact Recovery of a Hidden Community
Authors	Bruce Hajek, Yihong Wu, Jiaming Xu
Abstract	We study a semidefinite programming (SDP) relaxation of the maximum likelihood estimation for exactly recovering a hidden community of cardinality $K$ from an $n \times n$ symmetric data matrix $A$, where for distinct indices $i,j$, $A_{ij} \sim P$ if $i, j$ are both in the community and $A_{ij} \sim Q$ otherwise, for two known probability distributions $P$ and $Q$. We identify a sufficient condition and a necessary condition for the success of SDP for the general model. For both the Bernoulli case ($P={{\rm Bern}}(p)$ and $Q={{\rm Bern}}(q)$ with $p>q$) and the Gaussian case ($P=\mathcal{N}(\mu,1)$ and $Q=\mathcal{N}(0,1)$ with $\mu>0$), which correspond to the problem of planted dense subgraph recovery and submatrix localization respectively, the general results lead to the following findings: (1) If $K=\omega( n /\log n)$, SDP attains the information-theoretic recovery limits with sharp constants; (2) If $K=\Theta(n/\log n)$, SDP is order-wise optimal, but strictly suboptimal by a constant factor; (3) If $K=o(n/\log n)$ and $K \to \infty$, SDP is order-wise suboptimal. The same critical scaling for $K$ is found to hold, up to constant factors, for the performance of SDP on the stochastic block model of $n$ vertices partitioned into multiple communities of equal size $K$. A key ingredient in the proof of the necessary condition is a construction of a primal feasible solution based on random perturbation of the true cluster matrix.
Tasks
Published	2016-02-20
URL	http://arxiv.org/abs/1602.06410v2
PDF	http://arxiv.org/pdf/1602.06410v2.pdf
PWC	https://paperswithcode.com/paper/semidefinite-programs-for-exact-recovery-of-a
Repo
Framework

Stacking machine learning classifiers to identify Higgs bosons at the LHC


Title	Stacking machine learning classifiers to identify Higgs bosons at the LHC
Authors	Alexandre Alves
Abstract	Machine learning (ML) algorithms have been employed in the problem of classifying signal and background events with high accuracy in particle physics. In this paper, we compare the performance of a widespread ML technique, namely, \emph{stacked generalization}, against the results of two state-of-art algorithms: (1) a deep neural network (DNN) in the task of discovering a new neutral Higgs boson and (2) a scalable machine learning system for tree boosting, in the Standard Model Higgs to tau leptons channel, both at the 8 TeV LHC. In a cut-and-count analysis, \emph{stacking} three algorithms performed around 16% worse than DNN but demanding far less computation efforts, however, the same \emph{stacking} outperforms boosted decision trees. Using the stacked classifiers in a multivariate statistical analysis (MVA), on the other hand, significantly enhances the statistical significance compared to cut-and-count in both Higgs processes, suggesting that combining an ensemble of simpler and faster ML algorithms with MVA tools is a better approach than building a complex state-of-art algorithm for cut-and-count.
Tasks
Published	2016-12-21
URL	http://arxiv.org/abs/1612.07725v3
PDF	http://arxiv.org/pdf/1612.07725v3.pdf
PWC	https://paperswithcode.com/paper/stacking-machine-learning-classifiers-to
Repo
Framework

Moving Target Defense for Web Applications using Bayesian Stackelberg Games


Title	Moving Target Defense for Web Applications using Bayesian Stackelberg Games
Authors	Sailik Sengupta, Satya Gautam Vadlamudi, Subbarao Kambhampati, Marthony Taguinod, Adam Doupé, Ziming Zhao, Gail-Joon Ahn
Abstract	The present complexity in designing web applications makes software security a difficult goal to achieve. An attacker can explore a deployed service on the web and attack at his/her own leisure. Moving Target Defense (MTD) in web applications is an effective mechanism to nullify this advantage of their reconnaissance but the framework demands a good switching strategy when switching between multiple configurations for its web-stack. To address this issue, we propose modeling of a real-world MTD web application as a repeated Bayesian game. We then formulate an optimization problem that generates an effective switching strategy while considering the cost of switching between different web-stack configurations. To incorporate this model into a developed MTD system, we develop an automated system for generating attack sets of Common Vulnerabilities and Exposures (CVEs) for input attacker types with predefined capabilities. Our framework obtains realistic reward values for the players (defenders and attackers) in this game by using security domain expertise on CVEs obtained from the National Vulnerability Database (NVD). We also address the issue of prioritizing vulnerabilities that when fixed, improves the security of the MTD system. Lastly, we demonstrate the robustness of our proposed model by evaluating its performance when there is uncertainty about input attacker information.
Tasks
Published	2016-02-23
URL	http://arxiv.org/abs/1602.07024v3
PDF	http://arxiv.org/pdf/1602.07024v3.pdf
PWC	https://paperswithcode.com/paper/moving-target-defense-for-web-applications
Repo
Framework

A Mutual Contamination Analysis of Mixed Membership and Partial Label Models


Title	A Mutual Contamination Analysis of Mixed Membership and Partial Label Models
Authors	Julian Katz-Samuels, Clayton Scott
Abstract	Many machine learning problems can be characterized by mutual contamination models. In these problems, one observes several random samples from different convex combinations of a set of unknown base distributions. It is of interest to decontaminate mutual contamination models, i.e., to recover the base distributions either exactly or up to a permutation. This paper considers the general setting where the base distributions are defined on arbitrary probability spaces. We examine the decontamination problem in two mutual contamination models that describe popular machine learning tasks: recovering the base distributions up to a permutation in a mixed membership model, and recovering the base distributions exactly in a partial label model for classification. We give necessary and sufficient conditions for identifiability of both mutual contamination models, algorithms for both problems in the infinite and finite sample cases, and introduce novel proof techniques based on affine geometry.
Tasks
Published	2016-02-19
URL	http://arxiv.org/abs/1602.06235v1
PDF	http://arxiv.org/pdf/1602.06235v1.pdf
PWC	https://paperswithcode.com/paper/a-mutual-contamination-analysis-of-mixed
Repo
Framework

Generalization bound for kernel similarity learning


Title	Generalization bound for kernel similarity learning
Authors	Michael Rabadi
Abstract	Similarity learning has received a large amount of interest and is an important tool for many scientific and industrial applications. In this framework, we wish to infer the distance (similarity) between points with respect to an arbitrary distance function $d$. Here, we formulate the problem as a regression from a feature space $\mathcal{X}$ to an arbitrary vector space $\mathcal{Y}$, where the Euclidean distance is proportional to $d$. We then give Rademacher complexity bounds on the generalization error. We find that with high probability, the complexity is bounded by the maximum of the radius of $\mathcal{X}$ and the radius of $\mathcal{Y}$.
Tasks
Published	2016-10-12
URL	http://arxiv.org/abs/1610.03899v2
PDF	http://arxiv.org/pdf/1610.03899v2.pdf
PWC	https://paperswithcode.com/paper/generalization-bound-for-kernel-similarity
Repo
Framework

Application Specific Instrumentation (ASIN): A Bio-inspired Paradigm to Instrumentation using recognition before detection


Title	Application Specific Instrumentation (ASIN): A Bio-inspired Paradigm to Instrumentation using recognition before detection
Authors	Amit Kumar Mishra
Abstract	In this paper we present a new scheme for instrumentation, which has been inspired by the way small mammals sense their environment. We call this scheme Application Specific Instrumentation (ASIN). A conventional instrumentation system focuses on gathering as much information about the scene as possible. This, usually, is a generic system whose data can be used by another system to take a specific action. ASIN fuses these two steps into one. The major merit of the proposed scheme is that it uses low resolution sensors and much less computational overhead to give good performance for a highly specialised application
Tasks
Published	2016-10-31
URL	http://arxiv.org/abs/1611.00228v1
PDF	http://arxiv.org/pdf/1611.00228v1.pdf
PWC	https://paperswithcode.com/paper/application-specific-instrumentation-asin-a
Repo
Framework

A Deep Hierarchical Approach to Lifelong Learning in Minecraft


Title	A Deep Hierarchical Approach to Lifelong Learning in Minecraft
Authors	Chen Tessler, Shahar Givony, Tom Zahavy, Daniel J. Mankowitz, Shie Mannor
Abstract	We propose a lifelong learning system that has the ability to reuse and transfer knowledge from one task to another while efficiently retaining the previously learned knowledge-base. Knowledge is transferred by learning reusable skills to solve tasks in Minecraft, a popular video game which is an unsolved and high-dimensional lifelong learning problem. These reusable skills, which we refer to as Deep Skill Networks, are then incorporated into our novel Hierarchical Deep Reinforcement Learning Network (H-DRLN) architecture using two techniques: (1) a deep skill array and (2) skill distillation, our novel variation of policy distillation (Rusu et. al. 2015) for learning skills. Skill distillation enables the HDRLN to efficiently retain knowledge and therefore scale in lifelong learning, by accumulating knowledge and encapsulating multiple reusable skills into a single distilled network. The H-DRLN exhibits superior performance and lower learning sample complexity compared to the regular Deep Q Network (Mnih et. al. 2015) in sub-domains of Minecraft.
Tasks
Published	2016-04-25
URL	http://arxiv.org/abs/1604.07255v3
PDF	http://arxiv.org/pdf/1604.07255v3.pdf
PWC	https://paperswithcode.com/paper/a-deep-hierarchical-approach-to-lifelong
Repo
Framework

Consciousness is Pattern Recognition


Title	Consciousness is Pattern Recognition
Authors	Ray Van De Walker
Abstract	This is a proof of the strong AI hypothesis, i.e. that machines can be conscious. It is a phenomenological proof that pattern-recognition and subjective consciousness are the same activity in different terms. Therefore, it proves that essential subjective processes of consciousness are computable, and identifies significant traits and requirements of a conscious system. Since Husserl, many philosophers have accepted that consciousness consists of memories of logical connections between an ego and external objects. These connections are called “intentions.” Pattern recognition systems are achievable technical artifacts. The proof links this respected introspective philosophical theory of consciousness with technical art. The proof therefore endorses the strong AI hypothesis and may therefore also enable a theoretically-grounded form of artificial intelligence called a “synthetic intentionality,” able to synthesize, generalize, select and repeat intentions. If the pattern recognition is reflexive, able to operate on the set of intentions, and flexible, with several methods of synthesizing intentions, an SI may be a particularly strong form of AI. Similarities and possible applications to several AI paradigms are discussed. The article then addresses some problems: The proof’s limitations, reflexive cognition, Searles’ Chinese room, and how an SI could “understand” “meanings” and “be creative.”
Tasks
Published	2016-05-04
URL	http://arxiv.org/abs/1605.03009v2
PDF	http://arxiv.org/pdf/1605.03009v2.pdf
PWC	https://paperswithcode.com/paper/consciousness-is-pattern-recognition
Repo
Framework

Diagnosis of aerospace structure defects by a HPC implemented soft computing algorithm


Title	Diagnosis of aerospace structure defects by a HPC implemented soft computing algorithm
Authors	Gianni D’Angelo, Salvatore Rampone
Abstract	This study concerns with the diagnosis of aerospace structure defects by applying a HPC parallel implementation of a novel learning algorithm, named U-BRAIN. The Soft Computing approach allows advanced multi-parameter data processing in composite materials testing. The HPC parallel implementation overcomes the limits due to the great amount of data and the complexity of data processing. Our experimental results illustrate the effectiveness of the U-BRAIN parallel implementation as defect classifier in aerospace structures. The resulting system is implemented on a Linux-based cluster with multi-core architecture.
Tasks
Published	2016-10-18
URL	http://arxiv.org/abs/1610.05521v1
PDF	http://arxiv.org/pdf/1610.05521v1.pdf
PWC	https://paperswithcode.com/paper/diagnosis-of-aerospace-structure-defects-by-a
Repo
Framework

A simple and provable algorithm for sparse diagonal CCA


Title	A simple and provable algorithm for sparse diagonal CCA
Authors	Megasthenis Asteris, Anastasios Kyrillidis, Oluwasanmi Koyejo, Russell Poldrack
Abstract	Given two sets of variables, derived from a common set of samples, sparse Canonical Correlation Analysis (CCA) seeks linear combinations of a small number of variables in each set, such that the induced canonical variables are maximally correlated. Sparse CCA is NP-hard. We propose a novel combinatorial algorithm for sparse diagonal CCA, i.e., sparse CCA under the additional assumption that variables within each set are standardized and uncorrelated. Our algorithm operates on a low rank approximation of the input data and its computational complexity scales linearly with the number of input variables. It is simple to implement, and parallelizable. In contrast to most existing approaches, our algorithm administers precise control on the sparsity of the extracted canonical vectors, and comes with theoretical data-dependent global approximation guarantees, that hinge on the spectrum of the input data. Finally, it can be straightforwardly adapted to other constrained variants of CCA enforcing structure beyond sparsity. We empirically evaluate the proposed scheme and apply it on a real neuroimaging dataset to investigate associations between brain activity and behavior measurements.
Tasks
Published	2016-05-29
URL	http://arxiv.org/abs/1605.08961v1
PDF	http://arxiv.org/pdf/1605.08961v1.pdf
PWC	https://paperswithcode.com/paper/a-simple-and-provable-algorithm-for-sparse
Repo
Framework

Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery


Title	Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery
Authors	Zhiwei Li, Huanfeng Shen, Huifang Li, Guisong Xia, Paolo Gamba, Liangpei Zhang
Abstract	The wide field of view (WFV) imaging system onboard the Chinese GaoFen-1 (GF-1) optical satellite has a 16-m resolution and four-day revisit cycle for large-scale Earth observation. The advantages of the high temporal-spatial resolution and the wide field of view make the GF-1 WFV imagery very popular. However, cloud cover is an inevitable problem in GF-1 WFV imagery, which influences its precise application. Accurate cloud and cloud shadow detection in GF-1 WFV imagery is quite difficult due to the fact that there are only three visible bands and one near-infrared band. In this paper, an automatic multi-feature combined (MFC) method is proposed for cloud and cloud shadow detection in GF-1 WFV imagery. The MFC algorithm first implements threshold segmentation based on the spectral features and mask refinement based on guided filtering to generate a preliminary cloud mask. The geometric features are then used in combination with the texture features to improve the cloud detection results and produce the final cloud mask. Finally, the cloud shadow mask can be acquired by means of the cloud and shadow matching and follow-up correction process. The method was validated using 108 globally distributed scenes. The results indicate that MFC performs well under most conditions, and the average overall accuracy of MFC cloud detection is as high as 96.8%. In the contrastive analysis with the official provided cloud fractions, MFC shows a significant improvement in cloud fraction estimation, and achieves a high accuracy for the cloud and cloud shadow detection in the GF-1 WFV imagery with fewer spectral bands. The proposed method could be used as a preprocessing step in the future to monitor land-cover change, and it could also be easily extended to other optical satellite imagery which has a similar spectral setting.
Tasks	Cloud Detection, Shadow Detection
Published	2016-06-17
URL	http://arxiv.org/abs/1606.05415v4
PDF	http://arxiv.org/pdf/1606.05415v4.pdf
PWC	https://paperswithcode.com/paper/multi-feature-combined-cloud-and-cloud-shadow
Repo
Framework

Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation


Title	Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation
Authors	Dong Yu, Morten Kolbæk, Zheng-Hua Tan, Jesper Jensen
Abstract	We propose a novel deep learning model, which supports permutation invariant training (PIT), for speaker independent multi-talker speech separation, commonly known as the cocktail-party problem. Different from most of the prior arts that treat speech separation as a multi-class regression problem and the deep clustering technique that considers it a segmentation (or clustering) problem, our model optimizes for the separation regression error, ignoring the order of mixing sources. This strategy cleverly solves the long-lasting label permutation problem that has prevented progress on deep learning based techniques for speech separation. Experiments on the equal-energy mixing setup of a Danish corpus confirms the effectiveness of PIT. We believe improvements built upon PIT can eventually solve the cocktail-party problem and enable real-world adoption of, e.g., automatic meeting transcription and multi-party human-computer interaction, where overlapping speech is common.
Tasks	Speech Separation
Published	2016-07-01
URL	http://arxiv.org/abs/1607.00325v2
PDF	http://arxiv.org/pdf/1607.00325v2.pdf
PWC	https://paperswithcode.com/paper/permutation-invariant-training-of-deep-models
Repo
Framework