May 5, 2019

3280 words 16 mins read

Paper Group ANR 476

Multiplier-less Artificial Neurons Exploiting Error Resiliency for Energy-Efficient Neural Computing. Decision Making Based on Cohort Scores for Speaker Verification. Quantum perceptron over a field and neural network architecture selection in a quantum computer. Secure Multi-Party Computation Based Privacy Preserving Extreme Learning Machine Algor …

Multiplier-less Artificial Neurons Exploiting Error Resiliency for Energy-Efficient Neural Computing


Title	Multiplier-less Artificial Neurons Exploiting Error Resiliency for Energy-Efficient Neural Computing
Authors	Syed Shakib Sarwar, Swagath Venkataramani, Anand Raghunathan, Kaushik Roy
Abstract	Large-scale artificial neural networks have shown significant promise in addressing a wide range of classification and recognition applications. However, their large computational requirements stretch the capabilities of computing platforms. The fundamental components of these neural networks are the neurons and its synapses. The core of a digital hardware neuron consists of multiplier, accumulator and activation function. Multipliers consume most of the processing energy in the digital neurons, and thereby in the hardware implementations of artificial neural networks. We propose an approximate multiplier that utilizes the notion of computation sharing and exploits error resilience of neural network applications to achieve improved energy consumption. We also propose Multiplier-less Artificial Neuron (MAN) for even larger improvement in energy consumption and adapt the training process to ensure minimal degradation in accuracy. We evaluated the proposed design on 5 recognition applications. The results show, 35% and 60% reduction in energy consumption, for neuron sizes of 8 bits and 12 bits, respectively, with a maximum of ~2.83% loss in network accuracy, compared to a conventional neuron implementation. We also achieve 37% and 62% reduction in area for a neuron size of 8 bits and 12 bits, respectively, under iso-speed conditions.
Tasks
Published	2016-02-27
URL	http://arxiv.org/abs/1602.08557v1
PDF	http://arxiv.org/pdf/1602.08557v1.pdf
PWC	https://paperswithcode.com/paper/multiplier-less-artificial-neurons-exploiting
Repo
Framework

Decision Making Based on Cohort Scores for Speaker Verification


Title	Decision Making Based on Cohort Scores for Speaker Verification
Authors	Lantian Li, Renyu Wang, Gang Wang, Caixia Wang, Thomas Fang Zheng
Abstract	Decision making is an important component in a speaker verification system. For the conventional GMM-UBM architecture, the decision is usually conducted based on the log likelihood ratio of the test utterance against the GMM of the claimed speaker and the UBM. This single-score decision is simple but tends to be sensitive to the complex variations in speech signals (e.g. text content, channel, speaking style, etc.). In this paper, we propose a decision making approach based on multiple scores derived from a set of cohort GMMs (cohort scores). Importantly, these cohort scores are not simply averaged as in conventional cohort methods; instead, we employ a powerful discriminative model as the decision maker. Experimental results show that the proposed method delivers substantial performance improvement over the baseline system, especially when a deep neural network (DNN) is used as the decision maker, and the DNN input involves some statistical features derived from the cohort scores.
Tasks	Decision Making, Speaker Verification
Published	2016-09-27
URL	http://arxiv.org/abs/1609.08419v1
PDF	http://arxiv.org/pdf/1609.08419v1.pdf
PWC	https://paperswithcode.com/paper/decision-making-based-on-cohort-scores-for
Repo
Framework

Quantum perceptron over a field and neural network architecture selection in a quantum computer


Title	Quantum perceptron over a field and neural network architecture selection in a quantum computer
Authors	Adenilton J. da Silva, Teresa B. Ludermir, Wilson R. de Oliveira
Abstract	In this work, we propose a quantum neural network named quantum perceptron over a field (QPF). Quantum computers are not yet a reality and the models and algorithms proposed in this work cannot be simulated in actual (or classical) computers. QPF is a direct generalization of a classical perceptron and solves some drawbacks found in previous models of quantum perceptrons. We also present a learning algorithm named Superposition based Architecture Learning algorithm (SAL) that optimizes the neural network weights and architectures. SAL searches for the best architecture in a finite set of neural network architectures with linear time over the number of patterns in the training set. SAL is the first learning algorithm to determine neural network architectures in polynomial time. This speedup is obtained by the use of quantum parallelism and a non-linear quantum operator.
Tasks
Published	2016-01-29
URL	http://arxiv.org/abs/1602.00709v1
PDF	http://arxiv.org/pdf/1602.00709v1.pdf
PWC	https://paperswithcode.com/paper/quantum-perceptron-over-a-field-and-neural
Repo
Framework

Secure Multi-Party Computation Based Privacy Preserving Extreme Learning Machine Algorithm Over Vertically Distributed Data


Title	Secure Multi-Party Computation Based Privacy Preserving Extreme Learning Machine Algorithm Over Vertically Distributed Data
Authors	Ferhat Özgür Çatak
Abstract	Especially in the Big Data era, the usage of different classification methods is increasing day by day. The success of these classification methods depends on the effectiveness of learning methods. Extreme learning machine (ELM) classification algorithm is a relatively new learning method built on feed-forward neural-network. ELM classification algorithm is a simple and fast method that can create a model from high-dimensional data sets. Traditional ELM learning algorithm implicitly assumes complete access to whole data set. This is a major privacy concern in most of cases. Sharing of private data (i.e. medical records) is prevented because of security concerns. In this research, we propose an efficient and secure privacy-preserving learning algorithm for ELM classification over data that is vertically partitioned among several parties. The new learning method preserves the privacy on numerical attributes, builds a classification model without sharing private data without disclosing the data of each party to others.
Tasks
Published	2016-02-09
URL	http://arxiv.org/abs/1602.02899v1
PDF	http://arxiv.org/pdf/1602.02899v1.pdf
PWC	https://paperswithcode.com/paper/secure-multi-party-computation-based-privacy
Repo
Framework

A Confidence-Based Approach for Balancing Fairness and Accuracy


Title	A Confidence-Based Approach for Balancing Fairness and Accuracy
Authors	Benjamin Fish, Jeremy Kun, Ádám D. Lelkes
Abstract	We study three classical machine learning algorithms in the context of algorithmic fairness: adaptive boosting, support vector machines, and logistic regression. Our goal is to maintain the high accuracy of these learning algorithms while reducing the degree to which they discriminate against individuals because of their membership in a protected group. Our first contribution is a method for achieving fairness by shifting the decision boundary for the protected group. The method is based on the theory of margins for boosting. Our method performs comparably to or outperforms previous algorithms in the fairness literature in terms of accuracy and low discrimination, while simultaneously allowing for a fast and transparent quantification of the trade-off between bias and error. Our second contribution addresses the shortcomings of the bias-error trade-off studied in most of the algorithmic fairness literature. We demonstrate that even hopelessly naive modifications of a biased algorithm, which cannot be reasonably said to be fair, can still achieve low bias and high accuracy. To help to distinguish between these naive algorithms and more sensible algorithms we propose a new measure of fairness, called resilience to random bias (RRB). We demonstrate that RRB distinguishes well between our naive and sensible fairness algorithms. RRB together with bias and accuracy provides a more complete picture of the fairness of an algorithm.
Tasks
Published	2016-01-21
URL	http://arxiv.org/abs/1601.05764v1
PDF	http://arxiv.org/pdf/1601.05764v1.pdf
PWC	https://paperswithcode.com/paper/a-confidence-based-approach-for-balancing
Repo
Framework

Neural Taylor Approximations: Convergence and Exploration in Rectifier Networks


Title	Neural Taylor Approximations: Convergence and Exploration in Rectifier Networks
Authors	David Balduzzi, Brian McWilliams, Tony Butler-Yeoman
Abstract	Modern convolutional networks, incorporating rectifiers and max-pooling, are neither smooth nor convex; standard guarantees therefore do not apply. Nevertheless, methods from convex optimization such as gradient descent and Adam are widely used as building blocks for deep learning algorithms. This paper provides the first convergence guarantee applicable to modern convnets, which furthermore matches a lower bound for convex nonsmooth functions. The key technical tool is the neural Taylor approximation – a straightforward application of Taylor expansions to neural networks – and the associated Taylor loss. Experiments on a range of optimizers, layers, and tasks provide evidence that the analysis accurately captures the dynamics of neural optimization. The second half of the paper applies the Taylor approximation to isolate the main difficulty in training rectifier nets – that gradients are shattered – and investigates the hypothesis that, by exploring the space of activation configurations more thoroughly, adaptive optimizers such as RMSProp and Adam are able to converge to better solutions.
Tasks
Published	2016-11-07
URL	http://arxiv.org/abs/1611.02345v3
PDF	http://arxiv.org/pdf/1611.02345v3.pdf
PWC	https://paperswithcode.com/paper/neural-taylor-approximations-convergence-and
Repo
Framework

Clustering Based Feature Learning on Variable Stars


Title	Clustering Based Feature Learning on Variable Stars
Authors	Cristóbal Mackenzie, Karim Pichara, Pavlos Protopapas
Abstract	The success of automatic classification of variable stars strongly depends on the lightcurve representation. Usually, lightcurves are represented as a vector of many statistical descriptors designed by astronomers called features. These descriptors commonly demand significant computational power to calculate, require substantial research effort to develop and do not guarantee good performance on the final classification task. Today, lightcurve representation is not entirely automatic; algorithms that extract lightcurve features are designed by humans and must be manually tuned up for every survey. The vast amounts of data that will be generated in future surveys like LSST mean astronomers must develop analysis pipelines that are both scalable and automated. Recently, substantial efforts have been made in the machine learning community to develop methods that prescind from expert-designed and manually tuned features for features that are automatically learned from data. In this work we present what is, to our knowledge, the first unsupervised feature learning algorithm designed for variable stars. Our method first extracts a large number of lightcurve subsequences from a given set of photometric data, which are then clustered to find common local patterns in the time series. Representatives of these patterns, called exemplars, are then used to transform lightcurves of a labeled set into a new representation that can then be used to train an automatic classifier. The proposed algorithm learns the features from both labeled and unlabeled lightcurves, overcoming the bias generated when the learning process is done only with labeled data. We test our method on MACHO and OGLE datasets; the results show that the classification performance we achieve is as good and in some cases better than the performance achieved using traditional features, while the computational cost is significantly lower.
Tasks	Classification Of Variable Stars, Time Series
Published	2016-02-29
URL	http://arxiv.org/abs/1602.08977v1
PDF	http://arxiv.org/pdf/1602.08977v1.pdf
PWC	https://paperswithcode.com/paper/clustering-based-feature-learning-on-variable
Repo
Framework

FPGA system for real-time computational extended depth of field imaging using phase aperture coding


Title	FPGA system for real-time computational extended depth of field imaging using phase aperture coding
Authors	Tal Remez, Or Litany, Shachar Yoseff, Harel Haim, Alex Bronstein
Abstract	We present a proof-of-concept end-to-end system for computational extended depth of field (EDOF) imaging. The acquisition is performed through a phase-coded aperture implemented by placing a thin wavelength-dependent optical mask inside the pupil of a conventional camera lens, as a result of which, each color channel is focused at a different depth. The reconstruction process receives the raw Bayer image as the input, and performs blind estimation of the output color image in focus at an extended range of depths using a patch-wise sparse prior. We present a fast non-iterative reconstruction algorithm operating with constant latency in fixed-point arithmetics and achieving real-time performance in a prototype FPGA implementation. The output of the system, on simulated and real-life scenes, is qualitatively and quantitatively better than the result of clear-aperture imaging followed by state-of-the-art blind deblurring.
Tasks	Deblurring
Published	2016-08-03
URL	http://arxiv.org/abs/1608.01074v1
PDF	http://arxiv.org/pdf/1608.01074v1.pdf
PWC	https://paperswithcode.com/paper/fpga-system-for-real-time-computational
Repo
Framework

Machine Reading with Background Knowledge


Title	Machine Reading with Background Knowledge
Authors	Ndapandula Nakashole, Tom M. Mitchell
Abstract	Intelligent systems capable of automatically understanding natural language text are important for many artificial intelligence applications including mobile phone voice assistants, computer vision, and robotics. Understanding language often constitutes fitting new information into a previously acquired view of the world. However, many machine reading systems rely on the text alone to infer its meaning. In this paper, we pursue a different approach; machine reading methods that make use of background knowledge to facilitate language understanding. To this end, we have developed two methods: The first method addresses prepositional phrase attachment ambiguity. It uses background knowledge within a semi-supervised machine learning algorithm that learns from both labeled and unlabeled data. This approach yields state-of-the-art results on two datasets against strong baselines; The second method extracts relationships from compound nouns. Our knowledge-aware method for compound noun analysis accurately extracts relationships and significantly outperforms a baseline that does not make use of background knowledge.
Tasks	Prepositional Phrase Attachment, Reading Comprehension
Published	2016-12-16
URL	http://arxiv.org/abs/1612.05348v1
PDF	http://arxiv.org/pdf/1612.05348v1.pdf
PWC	https://paperswithcode.com/paper/machine-reading-with-background-knowledge
Repo
Framework

Applying Topological Persistence in Convolutional Neural Network for Music Audio Signals


Title	Applying Topological Persistence in Convolutional Neural Network for Music Audio Signals
Authors	Jen-Yu Liu, Shyh-Kang Jeng, Yi-Hsuan Yang
Abstract	Recent years have witnessed an increased interest in the application of persistent homology, a topological tool for data analysis, to machine learning problems. Persistent homology is known for its ability to numerically characterize the shapes of spaces induced by features or functions. On the other hand, deep neural networks have been shown effective in various tasks. To our best knowledge, however, existing neural network models seldom exploit shape information. In this paper, we investigate a way to use persistent homology in the framework of deep neural networks. Specifically, we propose to embed the so-called “persistence landscape,” a rather new topological summary for data, into a convolutional neural network (CNN) for dealing with audio signals. Our evaluation on automatic music tagging, a multi-label classification task, shows that the resulting persistent convolutional neural network (PCNN) model can perform significantly better than state-of-the-art models in prediction accuracy. We also discuss the intuition behind the design of the proposed model, and offer insights into the features that it learns.
Tasks	Multi-Label Classification
Published	2016-08-26
URL	http://arxiv.org/abs/1608.07373v1
PDF	http://arxiv.org/pdf/1608.07373v1.pdf
PWC	https://paperswithcode.com/paper/applying-topological-persistence-in
Repo
Framework

Video Summarization with Long Short-term Memory


Title	Video Summarization with Long Short-term Memory
Authors	Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman
Abstract	We propose a novel supervised learning technique for summarizing videos by automatically selecting keyframes or key subshots. Casting the problem as a structured prediction problem on sequential data, our main idea is to use Long Short-Term Memory (LSTM), a special type of recurrent neural networks to model the variable-range dependencies entailed in the task of video summarization. Our learning models attain the state-of-the-art results on two benchmark video datasets. Detailed analysis justifies the design of the models. In particular, we show that it is crucial to take into consideration the sequential structures in videos and model them. Besides advances in modeling techniques, we introduce techniques to address the need of a large number of annotated data for training complex learning models. There, our main idea is to exploit the existence of auxiliary annotated video datasets, albeit heterogeneous in visual styles and contents. Specifically, we show domain adaptation techniques can improve summarization by reducing the discrepancies in statistical properties across those datasets.
Tasks	Domain Adaptation, Structured Prediction, Video Summarization
Published	2016-05-26
URL	http://arxiv.org/abs/1605.08110v2
PDF	http://arxiv.org/pdf/1605.08110v2.pdf
PWC	https://paperswithcode.com/paper/video-summarization-with-long-short-term
Repo
Framework

Online Classification with Complex Metrics


Title	Online Classification with Complex Metrics
Authors	Bowei Yan, Oluwasanmi Koyejo, Kai Zhong, Pradeep Ravikumar
Abstract	We present a framework and analysis of consistent binary classification for complex and non-decomposable performance metrics such as the F-measure and the Jaccard measure. The proposed framework is general, as it applies to both batch and online learning, and to both linear and non-linear models. Our work follows recent results showing that the Bayes optimal classifier for many complex metrics is given by a thresholding of the conditional probability of the positive class. This manuscript extends this thresholding characterization – showing that the utility is strictly locally quasi-concave with respect to the threshold for a wide range of models and performance metrics. This, in turn, motivates simple normalized gradient ascent updates for threshold estimation. We present a finite-sample regret analysis for the resulting procedure. In particular, the risk for the batch case converges to the Bayes risk at the same rate as that of the underlying conditional probability estimation, and the risk of proposed online algorithm converges at a rate that depends on the conditional probability estimation risk. For instance, in the special case where the conditional probability model is logistic regression, our procedure achieves $O(\frac{1}{\sqrt{n}})$ sample complexity, both for batch and online training. Empirical evaluation shows that the proposed algorithms out-perform alternatives in practice, with comparable or better prediction performance and reduced run time for various metrics and datasets.
Tasks
Published	2016-10-23
URL	http://arxiv.org/abs/1610.07116v2
PDF	http://arxiv.org/pdf/1610.07116v2.pdf
PWC	https://paperswithcode.com/paper/online-classification-with-complex-metrics
Repo
Framework

Prediction of Infinite Words with Automata


Title	Prediction of Infinite Words with Automata
Authors	Tim Smith
Abstract	In the classic problem of sequence prediction, a predictor receives a sequence of values from an emitter and tries to guess the next value before it appears. The predictor masters the emitter if there is a point after which all of the predictor’s guesses are correct. In this paper we consider the case in which the predictor is an automaton and the emitted values are drawn from a finite set; i.e., the emitted sequence is an infinite word. We examine the predictive capabilities of finite automata, pushdown automata, stack automata (a generalization of pushdown automata), and multihead finite automata. We relate our predicting automata to purely periodic words, ultimately periodic words, and multilinear words, describing novel prediction algorithms for mastering these sequences.
Tasks
Published	2016-03-08
URL	http://arxiv.org/abs/1603.02597v1
PDF	http://arxiv.org/pdf/1603.02597v1.pdf
PWC	https://paperswithcode.com/paper/prediction-of-infinite-words-with-automata
Repo
Framework

Image Denoising Via Collaborative Support-Agnostic Recovery


Title	Image Denoising Via Collaborative Support-Agnostic Recovery
Authors	Muzammil Behzad, Mudassir Masood, Tarig Ballal, Maha Shadaydeh, Tareq Y. Al-Naffouri
Abstract	In this paper, we propose a novel image denoising algorithm using collaborative support-agnostic sparse reconstruction. An observed image is first divided into patches. Similarly structured patches are grouped together to be utilized for collaborative processing. In the proposed collaborative schemes, similar patches are assumed to share the same support taps. For sparse reconstruction, the likelihood of a tap being active in a patch is computed and refined through a collaboration process with other similar patches in the same group. This provides very good patch support estimation, hence enhancing the quality of image restoration. Performance comparisons with state-of-the-art algorithms, in terms of SSIM and PSNR, demonstrate the superiority of the proposed algorithm.
Tasks	Denoising, Image Denoising, Image Restoration
Published	2016-09-09
URL	http://arxiv.org/abs/1609.02932v1
PDF	http://arxiv.org/pdf/1609.02932v1.pdf
PWC	https://paperswithcode.com/paper/image-denoising-via-collaborative-support
Repo
Framework

Data Cleaning for XML Electronic Dictionaries via Statistical Anomaly Detection


Title	Data Cleaning for XML Electronic Dictionaries via Statistical Anomaly Detection
Authors	Michael Bloodgood, Benjamin Strauss
Abstract	Many important forms of data are stored digitally in XML format. Errors can occur in the textual content of the data in the fields of the XML. Fixing these errors manually is time-consuming and expensive, especially for large amounts of data. There is increasing interest in the research, development, and use of automated techniques for assisting with data cleaning. Electronic dictionaries are an important form of data frequently stored in XML format that frequently have errors introduced through a mixture of manual typographical entry errors and optical character recognition errors. In this paper we describe methods for flagging statistical anomalies as likely errors in electronic dictionaries stored in XML format. We describe six systems based on different sources of information. The systems detect errors using various signals in the data including uncommon characters, text length, character-based language models, word-based language models, tied-field length ratios, and tied-field transliteration models. Four of the systems detect errors based on expectations automatically inferred from content within elements of a single field type. We call these single-field systems. Two of the systems detect errors based on correspondence expectations automatically inferred from content within elements of multiple related field types. We call these tied-field systems. For each system, we provide an intuitive analysis of the type of error that it is successful at detecting. Finally, we describe two larger-scale evaluations using crowdsourcing with Amazon’s Mechanical Turk platform and using the annotations of a domain expert. The evaluations consistently show that the systems are useful for improving the efficiency with which errors in XML electronic dictionaries can be detected.
Tasks	Anomaly Detection, Optical Character Recognition, Transliteration
Published	2016-02-25
URL	http://arxiv.org/abs/1602.07807v2
PDF	http://arxiv.org/pdf/1602.07807v2.pdf
PWC	https://paperswithcode.com/paper/data-cleaning-for-xml-electronic-dictionaries
Repo
Framework