January 27, 2020

3421 words 17 mins read

Paper Group ANR 1146

Deep learning and face recognition: the state of the art. How to Incorporate Monotonicity in Deep Networks While Preserving Flexibility?. LSTM-Based Anomaly Detection: Detection Rules from Extreme Value Theory. Adversarial T-shirt! Evading Person Detectors in A Physical World. Fast generalization error bound of deep learning without scale invarianc …

Deep learning and face recognition: the state of the art


Title	Deep learning and face recognition: the state of the art
Authors	Stephen Balaban
Abstract	Deep Neural Networks (DNNs) have established themselves as a dominant technique in machine learning. DNNs have been top performers on a wide variety of tasks including image classification, speech recognition, and face recognition. Convolutional neural networks (CNNs) have been used in nearly all of the top performing methods on the Labeled Faces in the Wild (LFW) dataset. In this talk and accompanying paper, I attempt to provide a review and summary of the deep learning techniques used in the state-of-the-art. In addition, I highlight the need for both larger and more challenging public datasets to benchmark these systems. The high accuracy (99.63% for FaceNet at the time of publishing) and utilization of outside data (hundreds of millions of images in the case of Google’s FaceNet) suggest that current face verification benchmarks such as LFW may not be challenging enough, nor provide enough data, for current techniques. There exist a variety of organizations with mobile photo sharing applications that would be capable of releasing a very large scale and highly diverse dataset of facial images captured on mobile devices. Such an “ImageNet for Face Recognition” would likely receive a warm welcome from researchers and practitioners alike.
Tasks	Face Recognition, Face Verification, Image Classification, Speech Recognition
Published	2019-02-10
URL	http://arxiv.org/abs/1902.03524v1
PDF	http://arxiv.org/pdf/1902.03524v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-and-face-recognition-the-state
Repo
Framework

How to Incorporate Monotonicity in Deep Networks While Preserving Flexibility?


Title	How to Incorporate Monotonicity in Deep Networks While Preserving Flexibility?
Authors	Akhil Gupta, Naman Shukla, Lavanya Marla, Arinbjörn Kolbeinsson, Kartik Yellepeddi
Abstract	The importance of domain knowledge in enhancing model performance and making reliable predictions in the real-world is critical. This has led to an increased focus on specific model properties for interpretability. We focus on incorporating monotonic trends, and propose a novel gradient-based point-wise loss function for enforcing partial monotonicity with deep neural networks. While recent developments have relied on structural changes to the model, our approach aims at enhancing the learning process. Our model-agnostic point-wise loss function acts as a plug-in to the standard loss and penalizes non-monotonic gradients. We demonstrate that the point-wise loss produces comparable (and sometimes better) results on both AUC and monotonicity measure, as opposed to state-of-the-art deep lattice networks that guarantee monotonicity. Moreover, it is able to learn differentiated individual trends and produces smoother conditional curves which are important for personalized decisions, while preserving the flexibility of deep networks.
Tasks
Published	2019-09-24
URL	https://arxiv.org/abs/1909.10662v3
PDF	https://arxiv.org/pdf/1909.10662v3.pdf
PWC	https://paperswithcode.com/paper/monotonic-trends-in-deep-neural-networks
Repo
Framework

LSTM-Based Anomaly Detection: Detection Rules from Extreme Value Theory


Title	LSTM-Based Anomaly Detection: Detection Rules from Extreme Value Theory
Authors	Neema Davis, Gaurav Raina, Krishna Jagannathan
Abstract	In this paper, we explore various statistical techniques for anomaly detection in conjunction with the popular Long Short-Term Memory (LSTM) deep learning model for transportation networks. We obtain the prediction errors from an LSTM model, and then apply three statistical models based on (i) the Gaussian distribution, (ii) Extreme Value Theory (EVT), and (iii) the Tukey’s method. Using statistical tests and numerical studies, we find strong evidence against the widely employed Gaussian distribution based detection rule on the prediction errors. Next, motivated by fundamental results from Extreme Value Theory, we propose a detection technique that does not assume any parent distribution on the prediction errors. Through numerical experiments conducted on several real-world traffic data sets, we show that the EVT-based detection rule is superior to other detection rules, and is supported by statistical evidence.
Tasks	Anomaly Detection
Published	2019-09-13
URL	https://arxiv.org/abs/1909.06041v1
PDF	https://arxiv.org/pdf/1909.06041v1.pdf
PWC	https://paperswithcode.com/paper/lstm-based-anomaly-detection-detection-rules
Repo
Framework

Adversarial T-shirt! Evading Person Detectors in A Physical World


Title	Adversarial T-shirt! Evading Person Detectors in A Physical World
Authors	Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin-Yu Chen, Yanzhi Wang, Xue Lin
Abstract	It is known that deep neural networks (DNNs) are vulnerable to adversarial attacks. The so-called physical adversarial examples deceive DNN-based decision makers by attaching adversarial patches to real objects. However, most of the existing works on physical adversarial attacks focus on static objects such as glass frames, stop signs and images attached to cardboard. In this work, we propose Adversarial T-shirts, a robust physical adversarial example for evading person detectors even if it could undergo non-rigid deformation due to a moving person’s pose changes. To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to non-rigid objects such as T-shirts. We show that the proposed method achieves 74% and 57% attack success rates in digital and physical worlds respectively against YOLOv2. In contrast, the state-of-the-art physical attack method to fool a person detector only achieves 18% attack success rate. Furthermore, by leveraging min-max optimization, we extend our method to the ensemble attack setting against two object detectors YOLO-v2 and Faster R-CNN simultaneously.
Tasks
Published	2019-10-18
URL	https://arxiv.org/abs/1910.11099v2
PDF	https://arxiv.org/pdf/1910.11099v2.pdf
PWC	https://paperswithcode.com/paper/evading-real-time-person-detectors-by
Repo
Framework

Fast generalization error bound of deep learning without scale invariance of activation functions


Title	Fast generalization error bound of deep learning without scale invariance of activation functions
Authors	Yoshikazu Terada, Ryoma Hirose
Abstract	In theoretical analysis of deep learning, discovering which features of deep learning lead to good performance is an important task. In this paper, using the framework for analyzing the generalization error developed in Suzuki (2018), we derive a fast learning rate for deep neural networks with more general activation functions. In Suzuki (2018), assuming the scale invariance of activation functions, the tight generalization error bound of deep learning was derived. They mention that the scale invariance of the activation function is essential to derive tight error bounds. Whereas the rectified linear unit (ReLU; Nair and Hinton, 2010) satisfies the scale invariance, the other famous activation functions including the sigmoid and the hyperbolic tangent functions, and the exponential linear unit (ELU; Clevert et al., 2016) does not satisfy this condition. The existing analysis indicates a possibility that a deep learning with the non scale invariant activations may have a slower convergence rate of $O(1/\sqrt{n})$ when one with the scale invariant activations can reach a rate faster than $O(1/\sqrt{n})$. In this paper, without the scale invariance of activation functions, we derive the tight generalization error bound which is essentially the same as that of Suzuki (2018). From this result, at least in the framework of Suzuki (2018), it is shown that the scale invariance of the activation functions is not essential to get the fast rate of convergence. Simultaneously, it is also shown that the theoretical framework proposed by Suzuki (2018) can be widely applied for analysis of deep learning with general activation functions.
Tasks
Published	2019-07-25
URL	https://arxiv.org/abs/1907.10900v1
PDF	https://arxiv.org/pdf/1907.10900v1.pdf
PWC	https://paperswithcode.com/paper/fast-generalization-error-bound-of-deep
Repo
Framework

REVAMP$^2$T: Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking


Title	REVAMP$^2$T: Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking
Authors	Christopher Neff, Matías Mendieta, Shrey Mohan, Mohammadreza Baharani, Samuel Rogers, Hamed Tabkhi
Abstract	This article presents REVAMP$^2$T, Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking, as an integrated end-to-end IoT system for privacy-built-in decentralized situational awareness. REVAMP$^2$T presents novel algorithmic and system constructs to push deep learning and video analytics next to IoT devices (i.e. video cameras). On the algorithm side, REVAMP$^2$T proposes a unified integrated computer vision pipeline for detection, re-identification, and tracking across multiple cameras without the need for storing the streaming data. At the same time, it avoids facial recognition, and tracks and re-identifies pedestrians based on their key features at runtime. On the IoT system side, REVAMP$^2$T provides infrastructure to maximize hardware utilization on the edge, orchestrates global communications, and provides system-wide re-identification, without the use of personally identifiable information, for a distributed IoT network. For the results and evaluation, this article also proposes a new metric, Accuracy$\cdot$Efficiency (\AE), for holistic evaluation of IoT systems for real-time video analytics based on accuracy, performance, and power efficiency. REVAMP$^2$T outperforms current state-of-the-art by as much as thirteen-fold \AE~improvement.
Tasks
Published	2019-11-20
URL	https://arxiv.org/abs/1911.09217v2
PDF	https://arxiv.org/pdf/1911.09217v2.pdf
PWC	https://paperswithcode.com/paper/revamp2t-real-time-edge-video-analytics-for
Repo
Framework

Inductive Bias of Gradient Descent based Adversarial Training on Separable Data


Title	Inductive Bias of Gradient Descent based Adversarial Training on Separable Data
Authors	Yan Li, Ethan X. Fang, Huan Xu, Tuo Zhao
Abstract	Adversarial training is a principled approach for training robust neural networks. Despite of tremendous successes in practice, its theoretical properties still remain largely unexplored. In this paper, we provide new theoretical insights of gradient descent based adversarial training by studying its computational properties, specifically on its inductive bias. We take the binary classification task on linearly separable data as an illustrative example, where the loss asymptotically attains its infimum as the parameter diverges to infinity along certain directions. Specifically, we show that when the adversarial perturbation during training has bounded $\ell_2$-norm, the classifier learned by gradient descent based adversarial training converges in direction to the maximum $\ell_2$-norm margin classifier at the rate of $\tilde{\mathcal{O}}(1/\sqrt{T})$, significantly faster than the rate $\mathcal{O}(1/\log T)$ of training with clean data. In addition, when the adversarial perturbation during training has bounded $\ell_q$-norm for some $q\ge 1$, the resulting classifier converges in direction to a maximum mixed-norm margin classifier, which has a natural interpretation of robustness, as being the maximum $\ell_2$-norm margin classifier under worst-case $\ell_q$-norm perturbation to the data. Our findings provide theoretical backups for adversarial training that it indeed promotes robustness against adversarial perturbation.
Tasks
Published	2019-06-07
URL	https://arxiv.org/abs/1906.02931v3
PDF	https://arxiv.org/pdf/1906.02931v3.pdf
PWC	https://paperswithcode.com/paper/inductive-bias-of-gradient-descent-based
Repo
Framework

Formal Verification of Debates in Argumentation Theory


Title	Formal Verification of Debates in Argumentation Theory
Authors	Ria Jha, Francesco Belardinelli, Francesca Toni
Abstract	Humans engage in informal debates on a daily basis. By expressing their opinions and ideas in an argumentative fashion, they are able to gain a deeper understanding of a given problem and in some cases, find the best possible course of actions towards resolving it. In this paper, we develop a methodology to verify debates formalised as abstract argumentation frameworks. We first present a translation from debates to transition systems. Such transition systems can model debates and represent their evolution over time using a finite set of states. We then formalise relevant debate properties using temporal and strategy logics. These formalisations, along with a debate transition system, allow us to verify whether a given debate satisfies certain properties. The verification process can be automated using model checkers. Therefore, we also measure their performance when verifying debates, and use the results to discuss the feasibility of model checking debates.
Tasks	Abstract Argumentation
Published	2019-12-12
URL	https://arxiv.org/abs/1912.05828v1
PDF	https://arxiv.org/pdf/1912.05828v1.pdf
PWC	https://paperswithcode.com/paper/formal-verification-of-debates-in
Repo
Framework

A semi-holographic hyperdimensional representation system for hardware-friendly cognitive computing


Title	A semi-holographic hyperdimensional representation system for hardware-friendly cognitive computing
Authors	A. Serb, I. Kobyzev, J. Wang, T. Prodromakis
Abstract	One of the main, long-term objectives of artificial intelligence is the creation of thinking machines. To that end, substantial effort has been placed into designing cognitive systems; i.e. systems that can manipulate semantic-level information. A substantial part of that effort is oriented towards designing the mathematical machinery underlying cognition in a way that is very efficiently implementable in hardware. In this work we propose a ‘semi-holographic’ representation system that can be implemented in hardware using only multiplexing and addition operations, thus avoiding the need for expensive multiplication. The resulting architecture can be readily constructed by recycling standard microprocessor elements and is capable of performing two key mathematical operations frequently used in cognition, superposition and binding, within a budget of below 6 pJ for 64- bit operands. Our proposed ‘cognitive processing unit’ (CoPU) is intended as just one (albeit crucial) part of much larger cognitive systems where artificial neural networks of all kinds and associative memories work in concord to give rise to intelligence.
Tasks
Published	2019-07-12
URL	https://arxiv.org/abs/1907.05688v2
PDF	https://arxiv.org/pdf/1907.05688v2.pdf
PWC	https://paperswithcode.com/paper/a-semi-holographic-hyperdimensional
Repo
Framework

MineRL: A Large-Scale Dataset of Minecraft Demonstrations


Title	MineRL: A Large-Scale Dataset of Minecraft Demonstrations
Authors	William H. Guss, Brandon Houghton, Nicholay Topin, Phillip Wang, Cayden Codel, Manuela Veloso, Ruslan Salakhutdinov
Abstract	The sample inefficiency of standard deep reinforcement learning methods precludes their application to many real-world problems. Methods which leverage human demonstrations require fewer samples but have been researched less. As demonstrated in the computer vision and natural language processing communities, large-scale datasets have the capacity to facilitate research by serving as an experimental and benchmarking platform for new methods. However, existing datasets compatible with reinforcement learning simulators do not have sufficient scale, structure, and quality to enable the further development and evaluation of methods focused on using human examples. Therefore, we introduce a comprehensive, large-scale, simulator-paired dataset of human demonstrations: MineRL. The dataset consists of over 60 million automatically annotated state-action pairs across a variety of related tasks in Minecraft, a dynamic, 3D, open-world environment. We present a novel data collection scheme which allows for the ongoing introduction of new tasks and the gathering of complete state information suitable for a variety of methods. We demonstrate the hierarchality, diversity, and scale of the MineRL dataset. Further, we show the difficulty of the Minecraft domain along with the potential of MineRL in developing techniques to solve key research challenges within it.
Tasks
Published	2019-07-29
URL	https://arxiv.org/abs/1907.13440v1
PDF	https://arxiv.org/pdf/1907.13440v1.pdf
PWC	https://paperswithcode.com/paper/minerl-a-large-scale-dataset-of-minecraft
Repo
Framework

SCF2 – an Argumentation Semantics for Rational Human Judgments on Argument Acceptability: Technical Report


Title	SCF2 – an Argumentation Semantics for Rational Human Judgments on Argument Acceptability: Technical Report
Authors	Marcos Cramer, Leendert van der Torre
Abstract	In abstract argumentation theory, many argumentation semantics have been proposed for evaluating argumentation frameworks. This paper is based on the following research question: Which semantics corresponds well to what humans consider a rational judgment on the acceptability of arguments? There are two systematic ways to approach this research question: A normative perspective is provided by the principle-based approach, in which semantics are evaluated based on their satisfaction of various normatively desirable principles. A descriptive perspective is provided by the empirical approach, in which cognitive studies are conducted to determine which semantics best predicts human judgments about arguments. In this paper, we combine both approaches to motivate a new argumentation semantics called SCF2. For this purpose, we introduce and motivate two new principles and show that no semantics from the literature satisfies both of them. We define SCF2 and prove that it satisfies both new principles. Furthermore, we discuss findings of a recent empirical cognitive study that provide additional support to SCF2.
Tasks	Abstract Argumentation
Published	2019-08-22
URL	https://arxiv.org/abs/1908.08406v1
PDF	https://arxiv.org/pdf/1908.08406v1.pdf
PWC	https://paperswithcode.com/paper/scf2-an-argumentation-semantics-for-rational
Repo
Framework

Robust and Efficient Fuzzy C-Means Clustering Constrained on Flexible Sparsity


Title	Robust and Efficient Fuzzy C-Means Clustering Constrained on Flexible Sparsity
Authors	Jinglin Xu, Junwei Han, Mingliang Xu, Feiping Nie, Xuelong Li
Abstract	Clustering is an effective technique in data mining to group a set of objects in terms of some attributes. Among various clustering approaches, the family of K-Means algorithms gains popularity due to simplicity and efficiency. However, most of existing K-Means based clustering algorithms cannot deal with outliers well and are difficult to efficiently solve the problem embedded the $L_0$-norm constraint. To address the above issues and improve the performance of clustering significantly, we propose a novel clustering algorithm, named REFCMFS, which develops a $L_{2,1}$-norm robust loss as the data-driven item and imposes a $L_0$-norm constraint on the membership matrix to make the model more robust and sparse flexibly. In particular, REFCMFS designs a new way to simplify and solve the $L_0$-norm constraint without any approximate transformation by absorbing $\cdot_0$ into the objective function through a ranking function. These improvements not only make REFCMFS efficiently obtain more promising performance but also provide a new tractable and skillful optimization method to solve the problem embedded the $L_0$-norm constraint. Theoretical analyses and extensive experiments on several public datasets demonstrate the effectiveness and rationality of our proposed REFCMFS method.
Tasks
Published	2019-08-19
URL	https://arxiv.org/abs/1908.06699v4
PDF	https://arxiv.org/pdf/1908.06699v4.pdf
PWC	https://paperswithcode.com/paper/robust-and-efficient-fuzzy-c-means-clustering
Repo
Framework

Combining Unsupervised and Supervised Learning for Asset Class Failure Prediction in Power Systems


Title	Combining Unsupervised and Supervised Learning for Asset Class Failure Prediction in Power Systems
Authors	Ming Dong, L. S. Grumbach
Abstract	In power systems, an asset class is a group of power equipment that has the same function and shares similar electrical or mechanical characteristics. Predicting failures for different asset classes is critical for electric utilities towards developing cost-effective asset management strategies. Previously, physical age based Weibull distribution has been widely used to failure prediction. However, this mathematical model cannot incorporate asset condition data such as inspection or testing results. As a result, the prediction cannot be very specific and accurate for individual assets. To solve this important problem, this paper proposes a novel and comprehensive data-driven approach based on asset condition data: K-means clustering as an unsupervised learning method is used to analyze the inner structure of historical asset condition data and produce the asset conditional ages; logistic regression as a supervised learning method takes in both asset physical ages and conditional ages to classify and predict asset statuses. Furthermore, an index called average aging rate is defined to quantify, track and estimate the relationship between asset physical age and conditional age. This approach was applied to an urban distribution system in West Canada to predict medium-voltage cable failures. Case studies and comparison with standard Weibull distribution are provided. The proposed approach demonstrates superior performance and practicality for predicting asset class failures in power systems.
Tasks
Published	2019-01-06
URL	http://arxiv.org/abs/1901.01985v1
PDF	http://arxiv.org/pdf/1901.01985v1.pdf
PWC	https://paperswithcode.com/paper/combining-unsupervised-and-supervised
Repo
Framework

Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search


Title	Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search
Authors	Adnan Siraj Rakin, Zhezhi He, Deliang Fan
Abstract	Several important security issues of Deep Neural Network (DNN) have been raised recently associated with different applications and components. The most widely investigated security concern of DNN is from its malicious input, a.k.a adversarial example. Nevertheless, the security challenge of DNN’s parameters is not well explored yet. In this work, we are the first to propose a novel DNN weight attack methodology called Bit-Flip Attack (BFA) which can crush a neural network through maliciously flipping extremely small amount of bits within its weight storage memory system (i.e., DRAM). The bit-flip operations could be conducted through well-known Row-Hammer attack, while our main contribution is to develop an algorithm to identify the most vulnerable bits of DNN weight parameters (stored in memory as binary bits), that could maximize the accuracy degradation with a minimum number of bit-flips. Our proposed BFA utilizes a Progressive Bit Search (PBS) method which combines gradient ranking and progressive search to identify the most vulnerable bit to be flipped. With the aid of PBS, we can successfully attack a ResNet-18 fully malfunction (i.e., top-1 accuracy degrade from 69.8% to 0.1%) only through 13 bit-flips out of 93 million bits, while randomly flipping 100 bits merely degrades the accuracy by less than 1%.
Tasks
Published	2019-03-28
URL	http://arxiv.org/abs/1903.12269v2
PDF	http://arxiv.org/pdf/1903.12269v2.pdf
PWC	https://paperswithcode.com/paper/bit-flip-attack-crushing-neural-network
Repo
Framework

Leveraging Linguistic Characteristics for Bipolar Disorder Recognition with Gender Differences


Title	Leveraging Linguistic Characteristics for Bipolar Disorder Recognition with Gender Differences
Authors	Yen-Hao Huang, Yi-Hsin Chen, Fernando Henrique Calderon Alvarado, Ssu-Rui Lee, Shu-I Wu, Yuwen Lai, Yi-Shin Chen
Abstract	Most previous studies on automatic recognition model for bipolar disorder (BD) were based on both social media and linguistic features. The present study investigates the possibility of adopting only language-based features, namely the syntax and morpheme collocation. We also examine the effect of gender on the results considering gender has long been recognized as an important modulating factor for mental disorders, yet it received little attention in previous linguistic models. The present study collects Twitter posts 3 months prior to the self-disclosure by 349 BD users (231 female, 118 male). We construct a set of syntactic patterns in terms of the word usage based on graph pattern construction and pattern attention mechanism. The factors examined are gender differences, syntactic patterns, and bipolar recognition performance. The performance indicates our F1 scores reach over 91% and outperform several baselines, including those using TF-IDF, LIWC and pre-trained language models (ELMO and BERT). The contributions of the present study are: (1) The features are contextualized, domain-agnostic, and purely linguistic. (2) The performance of BD recognition is improved by gender-enriched linguistic pattern features, which are constructed with gender differences in language usage.
Tasks
Published	2019-07-17
URL	https://arxiv.org/abs/1907.07366v1
PDF	https://arxiv.org/pdf/1907.07366v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-linguistic-characteristics-for
Repo
Framework