October 20, 2019

3449 words 17 mins read

Paper Group ANR 43

Multi-Scale DenseNet-Based Electricity Theft Detection. Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks. Facial Attributes: Accuracy and Adversarial Robustness. PIRC Net : Using Proposal Indexing, Relationships and Context for Phrase Grounding. Inter-Scanner Harmonization of High Angular Resolution DW-MRI using Nul …

Multi-Scale DenseNet-Based Electricity Theft Detection


Title	Multi-Scale DenseNet-Based Electricity Theft Detection
Authors	Bo Li, Kele Xu, Xiaoyan Cui, Yiheng Wang, Xinbo Ai, Yanbo Wang
Abstract	Electricity theft detection issue has drawn lots of attention during last decades. Timely identification of the electricity theft in the power system is crucial for the safety and availability of the system. Although sustainable efforts have been made, the detection task remains challenging and falls short of accuracy and efficiency, especially with the increase of the data size. Recently, convolutional neural network-based methods have achieved better performance in comparison with traditional methods, which employ handcrafted features and shallow-architecture classifiers. In this paper, we present a novel approach for automatic detection by using a multi-scale dense connected convolution neural network (multi-scale DenseNet) in order to capture the long-term and short-term periodic features within the sequential data. We compare the proposed approaches with the classical algorithms, and the experimental results demonstrate that the multiscale DenseNet approach can significantly improve the accuracy of the detection. Moreover, our method is scalable, enabling larger data processing while no handcrafted feature engineering is needed.
Tasks	Feature Engineering
Published	2018-05-24
URL	http://arxiv.org/abs/1805.09591v1
PDF	http://arxiv.org/pdf/1805.09591v1.pdf
PWC	https://paperswithcode.com/paper/multi-scale-densenet-based-electricity-theft
Repo
Framework

Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks


Title	Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks
Authors	Nikola B. Kovachki, Andrew M. Stuart
Abstract	The standard probabilistic perspective on machine learning gives rise to empirical risk-minimization tasks that are frequently solved by stochastic gradient descent (SGD) and variants thereof. We present a formulation of these tasks as classical inverse or filtering problems and, furthermore, we propose an efficient, gradient-free algorithm for finding a solution to these problems using ensemble Kalman inversion (EKI). Applications of our approach include offline and online supervised learning with deep neural networks, as well as graph-based semi-supervised learning. The essence of the EKI procedure is an ensemble based approximate gradient descent in which derivatives are replaced by differences from within the ensemble. We suggest several modifications to the basic method, derived from empirically successful heuristics developed in the context of SGD. Numerical results demonstrate wide applicability and robustness of the proposed algorithm.
Tasks
Published	2018-08-10
URL	http://arxiv.org/abs/1808.03620v1
PDF	http://arxiv.org/pdf/1808.03620v1.pdf
PWC	https://paperswithcode.com/paper/ensemble-kalman-inversion-a-derivative-free
Repo
Framework

Facial Attributes: Accuracy and Adversarial Robustness


Title	Facial Attributes: Accuracy and Adversarial Robustness
Authors	Andras Rozsa, Manuel Günther, Ethan M. Rudd, Terrance E. Boult
Abstract	Facial attributes, emerging soft biometrics, must be automatically and reliably extracted from images in order to be usable in stand-alone systems. While recent methods extract facial attributes using deep neural networks (DNNs) trained on labeled facial attribute data, the robustness of deep attribute representations has not been evaluated. In this paper, we examine the representational stability of several approaches that recently advanced the state of the art on the CelebA benchmark by generating adversarial examples formed by adding small, non-random perturbations to inputs yielding altered classifications. We show that our fast flipping attribute (FFA) technique generates more adversarial examples than traditional algorithms, and that the adversarial robustness of DNNs varies highly between facial attributes. We also test the correlation of facial attributes and find that only for related attributes do the formed adversarial perturbations change the classification of others. Finally, we introduce the concept of natural adversarial samples, i.e., misclassified images where predictions can be corrected via small perturbations. We demonstrate that natural adversarial samples commonly occur and show that many of these images remain misclassified even with additional training epochs, even though their correct classification may require only a small adjustment to network parameters.
Tasks
Published	2018-01-04
URL	http://arxiv.org/abs/1801.02480v2
PDF	http://arxiv.org/pdf/1801.02480v2.pdf
PWC	https://paperswithcode.com/paper/facial-attributes-accuracy-and-adversarial
Repo
Framework

PIRC Net : Using Proposal Indexing, Relationships and Context for Phrase Grounding


Title	PIRC Net : Using Proposal Indexing, Relationships and Context for Phrase Grounding
Authors	Rama Kovvuri, Ram Nevatia
Abstract	Phrase Grounding aims to detect and localize objects in images that are referred to and are queried by natural language phrases. Phrase grounding finds applications in tasks such as Visual Dialog, Visual Search and Image-text co-reference resolution. In this paper, we present a framework that leverages information such as phrase category, relationships among neighboring phrases in a sentence and context to improve the performance of phrase grounding systems. We propose three modules: Proposal Indexing Network(PIN); Inter-phrase Regression Network(IRN) and Proposal Ranking Network(PRN) each of which analyze the region proposals of an image at increasing levels of detail by incorporating the above information. Also, in the absence of ground-truth spatial locations of the phrases(weakly-supervised), we propose knowledge transfer mechanisms that leverages the framework of PIN module. We demonstrate the effectiveness of our approach on the Flickr 30k Entities and ReferItGame datasets, for which we achieve improvements over state-of-the-art approaches in both supervised and weakly-supervised variants.
Tasks	Phrase Grounding, Transfer Learning, Visual Dialog
Published	2018-12-07
URL	http://arxiv.org/abs/1812.03213v1
PDF	http://arxiv.org/pdf/1812.03213v1.pdf
PWC	https://paperswithcode.com/paper/pirc-net-using-proposal-indexing
Repo
Framework

Inter-Scanner Harmonization of High Angular Resolution DW-MRI using Null Space Deep Learning


Title	Inter-Scanner Harmonization of High Angular Resolution DW-MRI using Null Space Deep Learning
Authors	Vishwesh Nath, Prasanna Parvathaneni, Colin B. Hansen, Allison E. Hainline, Camilo Bermudez, Samuel Remedios, Justin A. Blaber, Kurt G. Schilling, Ilwoo Lyu, Vaibhav Janve, Yurui Gao, Iwona Stepniewska, Baxter P. Rogers, Allen T. Newton, L. Taylor Davis, Jeff Luci, Adam W. Anderson, Bennett A. Landman
Abstract	Diffusion-weighted magnetic resonance imaging (DW-MRI) allows for non-invasive imaging of the local fiber architecture of the human brain at a millimetric scale. Multiple classical approaches have been proposed to detect both single (e.g., tensors) and multiple (e.g., constrained spherical deconvolution, CSD) fiber population orientations per voxel. However, existing techniques generally exhibit low reproducibility across MRI scanners. Herein, we propose a data-driven tech-nique using a neural network design which exploits two categories of data. First, training data were acquired on three squirrel monkey brains using ex-vivo DW-MRI and histology of the brain. Second, repeated scans of human subjects were acquired on two different scanners to augment the learning of the network pro-posed. To use these data, we propose a new network architecture, the null space deep network (NSDN), to simultaneously learn on traditional observed/truth pairs (e.g., MRI-histology voxels) along with repeated observations without a known truth (e.g., scan-rescan MRI). The NSDN was tested on twenty percent of the histology voxels that were kept completely blind to the network. NSDN significantly improved absolute performance relative to histology by 3.87% over CSD and 1.42% over a recently proposed deep neural network approach. More-over, it improved reproducibility on the paired data by 21.19% over CSD and 10.09% over a recently proposed deep approach. Finally, NSDN improved gen-eralizability of the model to a third in vivo human scanner (which was not used in training) by 16.08% over CSD and 10.41% over a recently proposed deep learn-ing approach. This work suggests that data-driven approaches for local fiber re-construction are more reproducible, informative and precise and offers a novel, practical method for determining these models.
Tasks
Published	2018-10-09
URL	http://arxiv.org/abs/1810.04260v1
PDF	http://arxiv.org/pdf/1810.04260v1.pdf
PWC	https://paperswithcode.com/paper/inter-scanner-harmonization-of-high-angular
Repo
Framework

Knowledge Aided Consistency for Weakly Supervised Phrase Grounding


Title	Knowledge Aided Consistency for Weakly Supervised Phrase Grounding
Authors	Kan Chen, Jiyang Gao, Ram Nevatia
Abstract	Given a natural language query, a phrase grounding system aims to localize mentioned objects in an image. In weakly supervised scenario, mapping between image regions (i.e., proposals) and language is not available in the training set. Previous methods address this deficiency by training a grounding system via learning to reconstruct language information contained in input queries from predicted proposals. However, the optimization is solely guided by the reconstruction loss from the language modality, and ignores rich visual information contained in proposals and useful cues from external knowledge. In this paper, we explore the consistency contained in both visual and language modalities, and leverage complementary external knowledge to facilitate weakly supervised grounding. We propose a novel Knowledge Aided Consistency Network (KAC Net) which is optimized by reconstructing input query and proposal’s information. To leverage complementary knowledge contained in the visual features, we introduce a Knowledge Based Pooling (KBP) gate to focus on query-related proposals. Experiments show that KAC Net provides a significant improvement on two popular datasets.
Tasks	Phrase Grounding
Published	2018-03-11
URL	http://arxiv.org/abs/1803.03879v1
PDF	http://arxiv.org/pdf/1803.03879v1.pdf
PWC	https://paperswithcode.com/paper/knowledge-aided-consistency-for-weakly
Repo
Framework

Composite Semantic Relation Classification


Title	Composite Semantic Relation Classification
Authors	Siamak Barzegar, Andre Freitas, Siegfried Handschuh, Brian Davis
Abstract	Different semantic interpretation tasks such as text entailment and question answering require the classification of semantic relations between terms or entities within text. However, in most cases it is not possible to assign a direct semantic relation between entities/terms. This paper proposes an approach for composite semantic relation classification, extending the traditional semantic relation classification task. Different from existing approaches, which use machine learning models built over lexical and distributional word vector features, the proposed model uses the combination of a large commonsense knowledge base of binary relations, a distributional navigational algorithm and sequence classification to provide a solution for the composite semantic relation classification problem.
Tasks	Question Answering, Relation Classification
Published	2018-05-16
URL	http://arxiv.org/abs/1805.06521v1
PDF	http://arxiv.org/pdf/1805.06521v1.pdf
PWC	https://paperswithcode.com/paper/composite-semantic-relation-classification
Repo
Framework

Dropout Distillation for Efficiently Estimating Model Confidence


Title	Dropout Distillation for Efficiently Estimating Model Confidence
Authors	Corina Gurau, Alex Bewley, Ingmar Posner
Abstract	We propose an efficient way to output better calibrated uncertainty scores from neural networks. The Distilled Dropout Network (DDN) makes standard (non-Bayesian) neural networks more introspective by adding a new training loss which prevents them from being overconfident. Our method is more efficient than Bayesian neural networks or model ensembles which, despite providing more reliable uncertainty scores, are more cumbersome to train and slower to test. We evaluate DDN on the the task of image classification on the CIFAR-10 dataset and show that our calibration results are competitive even when compared to 100 Monte Carlo samples from a dropout network while they also increase the classification accuracy. We also propose better calibration within the state of the art Faster R-CNN object detection framework and show, using the COCO dataset, that DDN helps train better calibrated object detectors.
Tasks	Calibration, Image Classification, Object Detection
Published	2018-09-27
URL	http://arxiv.org/abs/1809.10562v1
PDF	http://arxiv.org/pdf/1809.10562v1.pdf
PWC	https://paperswithcode.com/paper/dropout-distillation-for-efficiently
Repo
Framework

Convolutional Neural Networks Regularized by Correlated Noise


Title	Convolutional Neural Networks Regularized by Correlated Noise
Authors	Shamak Dutta, Bryan Tripp, Graham Taylor
Abstract	Neurons in the visual cortex are correlated in their variability. The presence of correlation impacts cortical processing because noise cannot be averaged out over many neurons. In an effort to understand the functional purpose of correlated variability, we implement and evaluate correlated noise models in deep convolutional neural networks. Inspired by the cortex, correlation is defined as a function of the distance between neurons and their selectivity. We show how to sample from high-dimensional correlated distributions while keeping the procedure differentiable, so that back-propagation can proceed as usual. The impact of correlated variability is evaluated on the classification of occluded and non-occluded images with and without the presence of other regularization techniques, such as dropout. More work is needed to understand the effects of correlations in various conditions, however in 10/12 of the cases we studied, the best performance on occluded images was obtained from a model with correlated noise.
Tasks
Published	2018-04-03
URL	http://arxiv.org/abs/1804.00815v1
PDF	http://arxiv.org/pdf/1804.00815v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-neural-networks-regularized-by
Repo
Framework

Blessing of dimensionality: mathematical foundations of the statistical physics of data


Title	Blessing of dimensionality: mathematical foundations of the statistical physics of data
Authors	A. N. Gorban, I. Y. Tyukin
Abstract	The concentration of measure phenomena were discovered as the mathematical background of statistical mechanics at the end of the XIX - beginning of the XX century and were then explored in mathematics of the XX-XXI centuries. At the beginning of the XXI century, it became clear that the proper utilisation of these phenomena in machine learning might transform the curse of dimensionality into the blessing of dimensionality. This paper summarises recently discovered phenomena of measure concentration which drastically simplify some machine learning problems in high dimension, and allow us to correct legacy artificial intelligence systems. The classical concentration of measure theorems state that i.i.d. random points are concentrated in a thin layer near a surface (a sphere or equators of a sphere, an average or median level set of energy or another Lipschitz function, etc.). The new stochastic separation theorems describe the thin structure of these thin layers: the random points are not only concentrated in a thin layer but are all linearly separable from the rest of the set, even for exponentially large random sets. The linear functionals for separation of points can be selected in the form of the linear Fisher’s discriminant. All artificial intelligence systems make errors. Non-destructive correction requires separation of the situations (samples) with errors from the samples corresponding to correct behaviour by a simple and robust classifier. The stochastic separation theorems provide us by such classifiers and a non-iterative (one-shot) procedure for learning.
Tasks
Published	2018-01-10
URL	http://arxiv.org/abs/1801.03421v1
PDF	http://arxiv.org/pdf/1801.03421v1.pdf
PWC	https://paperswithcode.com/paper/blessing-of-dimensionality-mathematical
Repo
Framework

Grounded Language Understanding for Manipulation Instructions Using GAN-Based Classification


Title	Grounded Language Understanding for Manipulation Instructions Using GAN-Based Classification
Authors	Komei Sugiura, Hisashi Kawai
Abstract	The target task of this study is grounded language understanding for domestic service robots (DSRs). In particular, we focus on instruction understanding for short sentences where verbs are missing. This task is of critical importance to build communicative DSRs because manipulation is essential for DSRs. Existing instruction understanding methods usually estimate missing information only from non-grounded knowledge; therefore, whether the predicted action is physically executable or not was unclear. In this paper, we present a grounded instruction understanding method to estimate appropriate objects given an instruction and situation. We extend the Generative Adversarial Nets (GAN) and build a GAN-based classifier using latent representations. To quantitatively evaluate the proposed method, we have developed a data set based on the standard data set used for Visual QA. Experimental results have shown that the proposed method gives the better result than baseline methods.
Tasks
Published	2018-01-16
URL	http://arxiv.org/abs/1801.05096v1
PDF	http://arxiv.org/pdf/1801.05096v1.pdf
PWC	https://paperswithcode.com/paper/grounded-language-understanding-for
Repo
Framework

PT-Spike: A Precise-Time-Dependent Single Spike Neuromorphic Architecture with Efficient Supervised Learning


Title	PT-Spike: A Precise-Time-Dependent Single Spike Neuromorphic Architecture with Efficient Supervised Learning
Authors	Tao Liu, Lei Jiang, Yier Jin, Gang Quan, Wujie Wen
Abstract	One of the most exciting advancements in AI over the last decade is the wide adoption of ANNs, such as DNN and CNN, in many real-world applications. However, the underlying massive amounts of computation and storage requirement greatly challenge their applicability in resource-limited platforms like the drone, mobile phone, and IoT devices etc. The third generation of neural network model–Spiking Neural Network (SNN), inspired by the working mechanism and efficiency of human brain, has emerged as a promising solution for achieving more impressive computing and power efficiency within light-weighted devices (e.g. single chip). However, the relevant research activities have been narrowly carried out on conventional rate-based spiking system designs for fulfilling the practical cognitive tasks, underestimating SNN’s energy efficiency, throughput, and system flexibility. Although the time-based SNN can be more attractive conceptually, its potentials are not unleashed in realistic applications due to lack of efficient coding and practical learning schemes. In this work, a Precise-Time-Dependent Single Spike Neuromorphic Architecture, namely “PT-Spike”, is developed to bridge this gap. Three constituent hardware-favorable techniques: precise single-spike temporal encoding, efficient supervised temporal learning, and fast asymmetric decoding are proposed accordingly to boost the energy efficiency and data processing capability of the time-based SNN at a more compact neural network model size when executing real cognitive tasks. Simulation results show that “PT-Spike” demonstrates significant improvements in network size, processing efficiency and power consumption with marginal classification accuracy degradation when compared with the rate-based SNN and ANN under the similar network configuration.
Tasks
Published	2018-03-14
URL	http://arxiv.org/abs/1803.05109v1
PDF	http://arxiv.org/pdf/1803.05109v1.pdf
PWC	https://paperswithcode.com/paper/pt-spike-a-precise-time-dependent-single
Repo
Framework

Functional Aggregate Queries with Additive Inequalities


Title	Functional Aggregate Queries with Additive Inequalities
Authors	Mahmoud Abo Khamis, Ryan R. Curtin, Benjamin Moseley, Hung Q. Ngo, XuanLong Nguyen, Dan Olteanu, Maximilian Schleich
Abstract	Motivated by fundamental applications in databases and relational machine learning, we formulate and study the problem of answering functional aggregate queries (FAQ) in which some of the input factors are defined by a collection of additive inequalities between variables. We refer to these queries as FAQ-AI for short. To answer FAQ-AI in the Boolean semiring, we define relaxed tree decompositions and relaxed submodular and fractional hypertree width parameters. We show that an extension of the InsideOut algorithm using Chazelle’s geometric data structure for solving the semigroup range search problem can answer Boolean FAQ-AI in time given by these new width parameters. This new algorithm achieves lower complexity than known solutions for FAQ-AI. It also recovers some known results in database query answering. Our second contribution is a relaxation of the set of polymatroids that gives rise to the counting version of the submodular width, denoted by #subw. This new width is sandwiched between the submodular and the fractional hypertree widths. Any FAQ and FAQ-AI over one semiring can be answered in time proportional to #subw and respectively to the relaxed version of #subw. We present three applications of our FAQ-AI framework to relational machine learning: k-means clustering, training linear support vector machines, and training models using non-polynomial loss. These optimization problems can be solved over a database asymptotically faster than computing the join of the database relations.
Tasks
Published	2018-12-22
URL	https://arxiv.org/abs/1812.09526v3
PDF	https://arxiv.org/pdf/1812.09526v3.pdf
PWC	https://paperswithcode.com/paper/on-functional-aggregate-queries-with-additive
Repo
Framework

Real-Time Referring Expression Comprehension by Single-Stage Grounding Network


Title	Real-Time Referring Expression Comprehension by Single-Stage Grounding Network
Authors	Xinpeng Chen, Lin Ma, Jingyuan Chen, Zequn Jie, Wei Liu, Jiebo Luo
Abstract	In this paper, we propose a novel end-to-end model, namely Single-Stage Grounding network (SSG), to localize the referent given a referring expression within an image. Different from previous multi-stage models which rely on object proposals or detected regions, our proposed model aims to comprehend a referring expression through one single stage without resorting to region proposals as well as the subsequent region-wise feature extraction. Specifically, a multimodal interactor is proposed to summarize the local region features regarding the referring expression attentively. Subsequently, a grounder is proposed to localize the referring expression within the given image directly. For further improving the localization accuracy, a guided attention mechanism is proposed to enforce the grounder to focus on the central region of the referent. Moreover, by exploiting and predicting visual attribute information, the grounder can further distinguish the referent objects within an image and thereby improve the model performance. Experiments on RefCOCO, RefCOCO+, and RefCOCOg datasets demonstrate that our proposed SSG without relying on any region proposals can achieve comparable performance with other advanced models. Furthermore, our SSG outperforms the previous models and achieves the state-of-art performance on the ReferItGame dataset. More importantly, our SSG is time efficient and can ground a referring expression in a 416416 image from the RefCOCO dataset in 25ms (40 referents per second) on average with a Nvidia Tesla P40, accomplishing more than 9 speedups over the existing multi-stage models.
Tasks
Published	2018-12-09
URL	http://arxiv.org/abs/1812.03426v1
PDF	http://arxiv.org/pdf/1812.03426v1.pdf
PWC	https://paperswithcode.com/paper/real-time-referring-expression-comprehension
Repo
Framework

Recognizing Partial Biometric Patterns


Title	Recognizing Partial Biometric Patterns
Authors	Lingxiao He, Zhenan Sun, Yuhao Zhu, Yunbo Wang
Abstract	Biometric recognition on partial captured targets is challenging, where only several partial observations of objects are available for matching. In this area, deep learning based methods are widely applied to match these partial captured objects caused by occlusions, variations of postures or just partial out of view in person re-identification and partial face recognition. However, most current methods are not able to identify an individual in case that some parts of the object are not obtainable, while the rest are specialized to certain constrained scenarios. To this end, we propose a robust general framework for arbitrary biometric matching scenarios without the limitations of alignment as well as the size of inputs. We introduce a feature post-processing step to handle the feature maps from FCN and a dictionary learning based Spatial Feature Reconstruction (SFR) to match different sized feature maps in this work. Moreover, the batch hard triplet loss function is applied to optimize the model. The applicability and effectiveness of the proposed method are demonstrated by the results from experiments on three person re-identification datasets (Market1501, CUHK03, DukeMTMC-reID), two partial person datasets (Partial REID and Partial iLIDS) and two partial face datasets (CASIA-NIR-Distance and Partial LFW), on which state-of-the-art performance is ensured in comparison with several state-of-the-art approaches. The code is released online and can be found on the website: https://github.com/lingxiao-he/Partial-Person-ReID.
Tasks	Dictionary Learning, Face Recognition, Person Re-Identification
Published	2018-10-17
URL	http://arxiv.org/abs/1810.07399v1
PDF	http://arxiv.org/pdf/1810.07399v1.pdf
PWC	https://paperswithcode.com/paper/recognizing-partial-biometric-patterns
Repo
Framework