October 18, 2019

2984 words 15 mins read

Paper Group ANR 522

NetScore: Towards Universal Metrics for Large-scale Performance Analysis of Deep Neural Networks for Practical On-Device Edge Usage. Partial AUC Maximization via Nonlinear Scoring Functions. A Psychopathological Approach to Safety Engineering in AI and AGI. Training Behavior of Sparse Neural Network Topologies. An Adaptive Learning Method of Deep B …

NetScore: Towards Universal Metrics for Large-scale Performance Analysis of Deep Neural Networks for Practical On-Device Edge Usage


Title	NetScore: Towards Universal Metrics for Large-scale Performance Analysis of Deep Neural Networks for Practical On-Device Edge Usage
Authors	Alexander Wong
Abstract	Much of the focus in the design of deep neural networks has been on improving accuracy, leading to more powerful yet highly complex network architectures that are difficult to deploy in practical scenarios, particularly on edge devices such as mobile and other consumer devices given their high computational and memory requirements. As a result, there has been a recent interest in the design of quantitative metrics for evaluating deep neural networks that accounts for more than just model accuracy as the sole indicator of network performance. In this study, we continue the conversation towards universal metrics for evaluating the performance of deep neural networks for practical on-device edge usage. In particular, we propose a new balanced metric called NetScore, which is designed specifically to provide a quantitative assessment of the balance between accuracy, computational complexity, and network architecture complexity of a deep neural network, which is important for on-device edge operation. In what is one of the largest comparative analysis between deep neural networks in literature, the NetScore metric, the top-1 accuracy metric, and the popular information density metric were compared across a diverse set of 60 different deep convolutional neural networks for image classification on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2012) dataset. The evaluation results across these three metrics for this diverse set of networks are presented in this study to act as a reference guide for practitioners in the field. The proposed NetScore metric, along with the other tested metrics, are by no means perfect, but the hope is to push the conversation towards better universal metrics for evaluating deep neural networks for use in practical on-device edge scenarios to help guide practitioners in model design for such scenarios.
Tasks	Image Classification, Object Recognition
Published	2018-06-14
URL	http://arxiv.org/abs/1806.05512v2
PDF	http://arxiv.org/pdf/1806.05512v2.pdf
PWC	https://paperswithcode.com/paper/netscore-towards-universal-metrics-for-large
Repo
Framework

Partial AUC Maximization via Nonlinear Scoring Functions


Title	Partial AUC Maximization via Nonlinear Scoring Functions
Authors	Naonori Ueda, Akinori Fujino
Abstract	We propose a method for maximizing a partial area under a receiver operating characteristic (ROC) curve (pAUC) for binary classification tasks. In binary classification tasks, accuracy is the most commonly used as a measure of classifier performance. In some applications such as anomaly detection and diagnostic testing, accuracy is not an appropriate measure since prior probabilties are often greatly biased. Although in such cases the pAUC has been utilized as a performance measure, few methods have been proposed for directly maximizing the pAUC. This optimization is achieved by using a scoring function. The conventional approach utilizes a linear function as the scoring function. In contrast we newly introduce nonlinear scoring functions for this purpose. Specifically, we present two types of nonlinear scoring functions based on generative models and deep neural networks. We show experimentally that nonlinear scoring fucntions improve the conventional methods through the application of a binary classification of real and bogus objects obtained with the Hyper Suprime-Cam on the Subaru telescope.
Tasks	Anomaly Detection
Published	2018-06-13
URL	http://arxiv.org/abs/1806.04838v1
PDF	http://arxiv.org/pdf/1806.04838v1.pdf
PWC	https://paperswithcode.com/paper/partial-auc-maximization-via-nonlinear
Repo
Framework

A Psychopathological Approach to Safety Engineering in AI and AGI


Title	A Psychopathological Approach to Safety Engineering in AI and AGI
Authors	Vahid Behzadan, Arslan Munir, Roman V. Yampolskiy
Abstract	The complexity of dynamics in AI techniques is already approaching that of complex adaptive systems, thus curtailing the feasibility of formal controllability and reachability analysis in the context of AI safety. It follows that the envisioned instances of Artificial General Intelligence (AGI) will also suffer from challenges of complexity. To tackle such issues, we propose the modeling of deleterious behaviors in AI and AGI as psychological disorders, thereby enabling the employment of psychopathological approaches to analysis and control of misbehaviors. Accordingly, we present a discussion on the feasibility of the psychopathological approaches to AI safety, and propose general directions for research on modeling, diagnosis, and treatment of psychological disorders in AGI.
Tasks
Published	2018-05-23
URL	http://arxiv.org/abs/1805.08915v1
PDF	http://arxiv.org/pdf/1805.08915v1.pdf
PWC	https://paperswithcode.com/paper/a-psychopathological-approach-to-safety
Repo
Framework

Training Behavior of Sparse Neural Network Topologies


Title	Training Behavior of Sparse Neural Network Topologies
Authors	Simon Alford, Ryan Robinett, Lauren Milechin, Jeremy Kepner
Abstract	Improvements in the performance of deep neural networks have often come through the design of larger and more complex networks. As a result, fast memory is a significant limiting factor in our ability to improve network performance. One approach to overcoming this limit is the design of sparse neural networks, which can be both very large and efficiently trained. In this paper we experiment training on sparse neural network topologies. We test pruning-based topologies, which are derived from an initially dense network whose connections are pruned, as well as RadiX-Nets, a class of network topologies with proven connectivity and sparsity properties. Results show that sparse networks obtain accuracies comparable to dense networks, but extreme levels of sparsity cause instability in training, which merits further study.
Tasks
Published	2018-09-30
URL	https://arxiv.org/abs/1810.00299v2
PDF	https://arxiv.org/pdf/1810.00299v2.pdf
PWC	https://paperswithcode.com/paper/pruned-and-structurally-sparse-neural
Repo
Framework

An Adaptive Learning Method of Deep Belief Network by Layer Generation Algorithm


Title	An Adaptive Learning Method of Deep Belief Network by Layer Generation Algorithm
Authors	Shin Kamada, Takumi Ichimura
Abstract	Deep Belief Network (DBN) has a deep architecture that represents multiple features of input patterns hierarchically with the pre-trained Restricted Boltzmann Machines (RBM). A traditional RBM or DBN model cannot change its network structure during the learning phase. Our proposed adaptive learning method can discover the optimal number of hidden neurons and weights and/or layers according to the input space. The model is an important method to take account of the computational cost and the model stability. The regularities to hold the sparse structure of network is considerable problem, since the extraction of explicit knowledge from the trained network should be required. In our previous research, we have developed the hybrid method of adaptive structural learning method of RBM and Learning Forgetting method to the trained RBM. In this paper, we propose the adaptive learning method of DBN that can determine the optimal number of layers during the learning. We evaluated our proposed model on some benchmark data sets.
Tasks
Published	2018-07-10
URL	http://arxiv.org/abs/1807.03486v2
PDF	http://arxiv.org/pdf/1807.03486v2.pdf
PWC	https://paperswithcode.com/paper/an-adaptive-learning-method-of-deep-belief
Repo
Framework

A non-convex approach to low-rank and sparse matrix decomposition


Title	A non-convex approach to low-rank and sparse matrix decomposition
Authors	Angang Cui, Meng Wen, Haiyang Li, Jigen Peng
Abstract	In this paper, we develop a nonconvex approach to the problem of low-rank and sparse matrix decomposition. In our nonconvex method, we replace the rank function and the $l_{0}$-norm of a given matrix with a non-convex fraction function on the singular values and the elements of the matrix respectively. An alternative direction method of multipliers algorithm is utilized to solve our proposed nonconvex problem with the nonconvex fraction function penalty. Numerical experiments on some low-rank and sparse matrix decomposition problems show that our method performs very well in recovering low-rank matrices which are heavily corrupted by large sparse errors.
Tasks
Published	2018-07-02
URL	https://arxiv.org/abs/1807.01276v2
PDF	https://arxiv.org/pdf/1807.01276v2.pdf
PWC	https://paperswithcode.com/paper/a-nonconvex-approach-to-low-rank-and-sparse
Repo
Framework

Gauges, Loops, and Polynomials for Partition Functions of Graphical Models


Title	Gauges, Loops, and Polynomials for Partition Functions of Graphical Models
Authors	Michael Chertkov, Vladimir Chernyak, Yury Maximov
Abstract	Graphical models (GM) represent multivariate and generally not normalized probability distributions. Computing the normalization factor, called the partition function (PF), is the main inference challenge relevant to multiple statistical and optimization applications. The problem is of an exponential complexity with respect to the number of variables. In this manuscript, aimed at approximating the PF, we consider Multi-Graph Models (MGMs) where binary variables and multivariable factors are associated with edges and nodes, respectively, of an undirected multi-graph. We suggest a new methodology for analysis and computations that combines the Gauge Function (GF) technique with the technique from the field of real stable polynomials. We show that the GF, representing a single-out term in a finite sum expression for the PF which achieves extremum at the so-called Belief-Propagation (BP) gauge, has a natural polynomial representation in terms of gauges/variables associated with edges of the multi-graph. Moreover, GF can be used to recover the PF through a sequence of transformations allowing appealing algebraic and graphical interpretations. Algebraically, one step in the sequence consists in application of a differential operator over gauges associated with an edge. Graphically, the sequence is interpreted as a repetitive elimination/contraction of edges resulting in MGMs on decreasing in size (number of edges) graphs with the same PF as in the original MGM. Even though complexity of computing factors in the sequence of derived MGMs and respective GFs grow exponentially with the number of eliminated edges, polynomials associated with the new factors remain bi-stable if the original factors have this property. Moreover, we show that BP estimations in the sequence do not decrease, each low-bounding the PF.
Tasks
Published	2018-11-12
URL	http://arxiv.org/abs/1811.04713v5
PDF	http://arxiv.org/pdf/1811.04713v5.pdf
PWC	https://paperswithcode.com/paper/gauges-loops-and-polynomials-for-partition
Repo
Framework

Preserving Semantic Relations for Zero-Shot Learning


Title	Preserving Semantic Relations for Zero-Shot Learning
Authors	Yashas Annadani, Soma Biswas
Abstract	Zero-shot learning has gained popularity due to its potential to scale recognition models without requiring additional training data. This is usually achieved by associating categories with their semantic information like attributes. However, we believe that the potential offered by this paradigm is not yet fully exploited. In this work, we propose to utilize the structure of the space spanned by the attributes using a set of relations. We devise objective functions to preserve these relations in the embedding space, thereby inducing semanticity to the embedding space. Through extensive experimental evaluation on five benchmark datasets, we demonstrate that inducing semanticity to the embedding space is beneficial for zero-shot learning. The proposed approach outperforms the state-of-the-art on the standard zero-shot setting as well as the more realistic generalized zero-shot setting. We also demonstrate how the proposed approach can be useful for making approximate semantic inferences about an image belonging to a category for which attribute information is not available.
Tasks	Zero-Shot Learning
Published	2018-03-08
URL	http://arxiv.org/abs/1803.03049v1
PDF	http://arxiv.org/pdf/1803.03049v1.pdf
PWC	https://paperswithcode.com/paper/preserving-semantic-relations-for-zero-shot
Repo
Framework

Error Analysis and Improving the Accuracy of Winograd Convolution for Deep Neural Networks


Title	Error Analysis and Improving the Accuracy of Winograd Convolution for Deep Neural Networks
Authors	Barbara Barabasz, Andrew Anderson, Kirk M. Soodhalter, David Gregg
Abstract	Popular deep neural networks (DNNs) spend the majority of their execution time computing convolutions. The Winograd family of algorithms can greatly reduce the number of arithmetic operations required and is present in many DNN software frameworks. However, the performance gain is at the expense of a reduction in floating point (FP) numerical accuracy. In this paper, we analyse the worst case FP error and prove the estimation of norm and conditioning of the algorithm. We show that the bound grows exponentially with the size of the convolution, but the error bound of the \textit{modified} algorithm is smaller than the original one. We propose several methods for reducing FP error. We propose a canonical evaluation ordering based on Huffman coding that reduces summation error. We study the selection of sampling “points” experimentally and find empirically good points for the most important sizes. We identify the main factors associated with good points. In addition, we explore other methods to reduce FP error, including mixed-precision convolution, and pairwise summation across DNN channels. Using our methods we can significantly reduce FP error for a given block size, which allows larger block sizes and reduced computation.
Tasks
Published	2018-03-29
URL	http://arxiv.org/abs/1803.10986v3
PDF	http://arxiv.org/pdf/1803.10986v3.pdf
PWC	https://paperswithcode.com/paper/error-analysis-and-improving-the-accuracy-of
Repo
Framework

Object Activity Scene Description, Construction and Recognition


Title	Object Activity Scene Description, Construction and Recognition
Authors	Hui Feng, Shanshan Wang, Shuzhi Sam Ge
Abstract	Action recognition is a critical task for social robots to meaningfully engage with their environment. 3D human skeleton-based action recognition is an attractive research area in recent years. Although, the existing approaches are good at action recognition, it is a great challenge to recognize a group of actions in an activity scene. To tackle this problem, at first, we partition the scene into several primitive actions (PAs) based upon motion attention mechanism. Then, the primitive actions are described by the trajectory vectors of corresponding joints. After that, motivated by text classification based on word embedding, we employ convolution neural network (CNN) to recognize activity scenes by considering motion of joints as “word” of activity. The experimental results on the scenes of human activity dataset show the efficiency of the proposed approach.
Tasks	Skeleton Based Action Recognition, Temporal Action Localization, Text Classification
Published	2018-05-01
URL	http://arxiv.org/abs/1805.00258v1
PDF	http://arxiv.org/pdf/1805.00258v1.pdf
PWC	https://paperswithcode.com/paper/object-activity-scene-description
Repo
Framework

A Formal Ontology-Based Classification of Lexemes and its Applications


Title	A Formal Ontology-Based Classification of Lexemes and its Applications
Authors	Sreekavitha Parupalli, Navjyoti Singh
Abstract	The paper describes the enrichment of OntoSenseNet - a verb-centric lexical resource for Indian Languages. A major contribution of this work is preservation of an authentic Telugu dictionary by developing a computational version of the same. It is important because native speakers can better annotate the sense-types when both the word and its meaning are in Telugu. Hence efforts are made to develop the aforementioned Telugu dictionary and annotations are done manually. The manually annotated gold standard corpus consists 8483 verbs, 253 adverbs and 1673 adjectives. Annotations are done by native speakers according to defined annotation guidelines. In this paper, we provide an overview of the annotation procedure and present the validation of the developed resource through inter-annotator agreement. Additional words from Telugu WordNet are added to our resource and are crowd-sourced for annotation. The statistics are compared with the sense-annotated lexicon, our resource for more insights.
Tasks
Published	2018-07-04
URL	http://arxiv.org/abs/1807.01996v1
PDF	http://arxiv.org/pdf/1807.01996v1.pdf
PWC	https://paperswithcode.com/paper/a-formal-ontology-based-classification-of
Repo
Framework

Ontology-Grounded Topic Modeling for Climate Science Research


Title	Ontology-Grounded Topic Modeling for Climate Science Research
Authors	Jennifer Sleeman, Tim Finin, Milton Halem
Abstract	In scientific disciplines where research findings have a strong impact on society, reducing the amount of time it takes to understand, synthesize and exploit the research is invaluable. Topic modeling is an effective technique for summarizing a collection of documents to find the main themes among them and to classify other documents that have a similar mixture of co-occurring words. We show how grounding a topic model with an ontology, extracted from a glossary of important domain phrases, improves the topics generated and makes them easier to understand. We apply and evaluate this method to the climate science domain. The result improves the topics generated and supports faster research understanding, discovery of social networks among researchers, and automatic ontology generation.
Tasks
Published	2018-07-28
URL	http://arxiv.org/abs/1807.10965v2
PDF	http://arxiv.org/pdf/1807.10965v2.pdf
PWC	https://paperswithcode.com/paper/ontology-grounded-topic-modeling-for-climate
Repo
Framework

PAD-Net: A Perception-Aided Single Image Dehazing Network


Title	PAD-Net: A Perception-Aided Single Image Dehazing Network
Authors	Yu Liu, Guanlong Zhao
Abstract	In this work, we investigate the possibility of replacing the $\ell_2$ loss with perceptually derived loss functions (SSIM, MS-SSIM, etc.) in training an end-to-end dehazing neural network. Objective experimental results suggest that by merely changing the loss function we can obtain significantly higher PSNR and SSIM scores on the SOTS set in the RESIDE dataset, compared with a state-of-the-art end-to-end dehazing neural network (AOD-Net) that uses the $\ell_2$ loss. The best PSNR we obtained was 23.50 (4.2% relative improvement), and the best SSIM we obtained was 0.8747 (2.3% relative improvement.)
Tasks	Image Dehazing, Single Image Dehazing
Published	2018-05-08
URL	http://arxiv.org/abs/1805.03146v1
PDF	http://arxiv.org/pdf/1805.03146v1.pdf
PWC	https://paperswithcode.com/paper/pad-net-a-perception-aided-single-image
Repo
Framework

A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation


Title	A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation
Authors	Holger R. Roth, Chen Shen, Hirohisa Oda, Takaaki Sugino, Masahiro Oda, Yuichiro Hayashi, Kazunari Misawa, Kensaku Mori
Abstract	Recent advances in deep learning, like 3D fully convolutional networks (FCNs), have improved the state-of-the-art in dense semantic segmentation of medical images. However, most network architectures require severely downsampling or cropping the images to meet the memory limitations of today’s GPU cards while still considering enough context in the images for accurate segmentation. In this work, we propose a novel approach that utilizes auto-context to perform semantic segmentation at higher resolutions in a multi-scale pyramid of stacked 3D FCNs. We train and validate our models on a dataset of manually annotated abdominal organs and vessels from 377 clinical CT images used in gastric surgery, and achieve promising results with close to 90% Dice score on average. For additional evaluation, we perform separate testing on datasets from different sources and achieve competitive results, illustrating the robustness of the model and approach.
Tasks	Semantic Segmentation
Published	2018-06-06
URL	http://arxiv.org/abs/1806.02237v1
PDF	http://arxiv.org/pdf/1806.02237v1.pdf
PWC	https://paperswithcode.com/paper/a-multi-scale-pyramid-of-3d-fully
Repo
Framework

Block Matching Frame based Material Reconstruction for Spectral CT


Title	Block Matching Frame based Material Reconstruction for Spectral CT
Authors	Weiwen Wu, Qian Wang, Fenglin Liu, Yining Zhu, Hengyong Yu
Abstract	Spectral computed tomography (CT) has a great potential in material identification and decomposition. To achieve high-quality material composition images and further suppress the x-ray beam hardening artifacts, we first propose a one-step material reconstruction model based on Taylor first-order expansion. Then, we develop a basic material reconstruction method named material simultaneous algebraic reconstruction technique (MSART). Considering the local similarity of each material image, we incorporate a powerful block matching frame (BMF) into the material reconstruction (MR) model and generate a BMF based MR (BMFMR) method. Because the BMFMR model contains the L0-norm problem, we adopt a split-Bregman method for optimization. The numerical simulation and physical phantom experiment results validate the correctness of the material reconstruction algorithms and demonstrate that the BMF regularization outperforms the total variation and no-local mean regularizations.
Tasks	Computed Tomography (CT)
Published	2018-10-22
URL	https://arxiv.org/abs/1810.10346v2
PDF	https://arxiv.org/pdf/1810.10346v2.pdf
PWC	https://paperswithcode.com/paper/block-matching-frame-based-material
Repo
Framework