Paper Group ANR 318
DoPa: A Comprehensive CNN Detection Methodology against Physical Adversarial Attacks. A Closed-Form Learned Pooling for Deep Classification Networks. Interpreting and Evaluating Neural Network Robustness. Construction of Macro Actions for Deep Reinforcement Learning. Discriminative Ridge Machine: A Classifier for High-Dimensional Data or Imbalanced …
DoPa: A Comprehensive CNN Detection Methodology against Physical Adversarial Attacks
| Title | DoPa: A Comprehensive CNN Detection Methodology against Physical Adversarial Attacks |
| Authors | Zirui Xu, Fuxun Yu, Xiang Chen |
| Abstract | Recently, Convolutional Neural Networks (CNNs) demonstrate a considerable vulnerability to adversarial attacks, which can be easily misled by adversarial perturbations. With more aggressive methods proposed, adversarial attacks can be also applied to the physical world, causing practical issues to various CNN powered applications. To secure CNNs, adversarial attack detection is considered as the most critical approach. However, most existing works focus on superficial patterns and merely search a particular method to differentiate the adversarial inputs and natural inputs, ignoring the analysis of CNN inner vulnerability. Therefore, they can only target to specific physical adversarial attacks, lacking expected versatility to different attacks. To address this issue, we propose DoPa – a comprehensive CNN detection methodology for various physical adversarial attacks. By interpreting the CNN’s vulnerability, we find that non-semantic adversarial perturbations can activate CNN with significantly abnormal activations and even overwhelm other semantic input patterns’ activations. Therefore, we add a self-verification stage to analyze the semantics of distinguished activation patterns, which improves the CNN recognition process. We apply such a detection methodology into both image and audio CNN recognition scenarios. Experiments show that DoPa can achieve an average rate of 90% success for image attack detection and 92% success for audio attack detection. Announcement:[The original DoPa draft on arXiv was modified and submitted to a conference already, while this short abstract was submitted only for a presentation at the KDD 2019 AIoT Workshop.] |
| Tasks | Adversarial Attack |
| Published | 2019-05-21 |
| URL | https://arxiv.org/abs/1905.08790v4 |
| https://arxiv.org/pdf/1905.08790v4.pdf | |
| PWC | https://paperswithcode.com/paper/dopa-a-fast-and-comprehensive-cnn-defense |
| Repo | |
| Framework | |
A Closed-Form Learned Pooling for Deep Classification Networks
| Title | A Closed-Form Learned Pooling for Deep Classification Networks |
| Authors | Vighnesh Birodkar, Hossein Mobahi, Dilip Krishnan, Samy Bengio |
| Abstract | In modern computer vision tasks, convolutional neural networks (CNNs) are indispensable for image classification tasks due to their efficiency and effectiveness. Part of their superiority compared to other architectures, comes from the fact that a single, local filter is shared across the entire image. However, there are scenarios where we may need to treat spatial locations in non-uniform manner. We see this in nature when considering how humans have evolved foveation to process different areas in their field of vision with varying levels of detail. In this paper we propose a way to enable CNNs to learn different pooling weights for each pixel location. We do so by introducing an extended definition of a pooling operator. This operator can learn a strict super-set of what can be learned by average pooling or convolutions. It has the benefit of being shared across feature maps and can be encouraged to be local or diffuse depending on the data. We show that for fixed network weights, our pooling operator can be computed in closed-form by spectral decomposition of matrices associated with class separability. Through experiments, we show that this operator benefits generalization for ResNets and CNNs on the CIFAR-10, CIFAR-100 and SVHN datasets and improves robustness to geometric corruptions and perturbations on the CIFAR-10-C and CIFAR-10-P test sets. |
| Tasks | Foveation, Image Classification |
| Published | 2019-06-10 |
| URL | https://arxiv.org/abs/1906.03808v1 |
| https://arxiv.org/pdf/1906.03808v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-closed-form-learned-pooling-for-deep |
| Repo | |
| Framework | |
Interpreting and Evaluating Neural Network Robustness
| Title | Interpreting and Evaluating Neural Network Robustness |
| Authors | Fuxun Yu, Zhuwei Qin, Chenchen Liu, Liang Zhao, Yanzhi Wang, Xiang Chen |
| Abstract | Recently, adversarial deception becomes one of the most considerable threats to deep neural networks. However, compared to extensive research in new designs of various adversarial attacks and defenses, the neural networks’ intrinsic robustness property is still lack of thorough investigation. This work aims to qualitatively interpret the adversarial attack and defense mechanism through loss visualization, and establish a quantitative metric to evaluate the neural network model’s intrinsic robustness. The proposed robustness metric identifies the upper bound of a model’s prediction divergence in the given domain and thus indicates whether the model can maintain a stable prediction. With extensive experiments, our metric demonstrates several advantages over conventional adversarial testing accuracy based robustness estimation: (1) it provides a uniformed evaluation to models with different structures and parameter scales; (2) it over-performs conventional accuracy based robustness estimation and provides a more reliable evaluation that is invariant to different test settings; (3) it can be fast generated without considerable testing cost. |
| Tasks | Adversarial Attack |
| Published | 2019-05-10 |
| URL | https://arxiv.org/abs/1905.04270v1 |
| https://arxiv.org/pdf/1905.04270v1.pdf | |
| PWC | https://paperswithcode.com/paper/interpreting-and-evaluating-neural-network |
| Repo | |
| Framework | |
Construction of Macro Actions for Deep Reinforcement Learning
| Title | Construction of Macro Actions for Deep Reinforcement Learning |
| Authors | Yi-Hsiang Chang, Kuan-Yu Chang, Henry Kuo, Chun-Yi Lee |
| Abstract | Conventional deep reinforcement learning typically determines an appropriate primitive action at each timestep, which requires enormous amount of time and effort for learning an effective policy, especially in large and complex environments. To deal with the issue fundamentally, we incorporate macro actions, defined as sequences of primitive actions, into the primitive action space to form an augmented action space. The problem lies in how to find an appropriate macro action to augment the primitive action space. The agent using a proper augmented action space is able to jump to a farther state and thus speed up the exploration process as well as facilitate the learning procedure. In previous researches, macro actions are developed by mining the most frequently used action sequences or repeating previous actions. However, the most frequently used action sequences are extracted from a past policy, which may only reinforce the original behavior of that policy. On the other hand, repeating actions may limit the diversity of behaviors of the agent. Instead, we propose to construct macro actions by a genetic algorithm, which eliminates the dependency of the macro action derivation procedure from the past policies of the agent. Our approach appends a macro action to the primitive action space once at a time and evaluates whether the augmented action space leads to promising performance or not. We perform extensive experiments and show that the constructed macro actions are able to speed up the learning process for a variety of deep reinforcement learning methods. Our experimental results also demonstrate that the macro actions suggested by our approach are transferable among deep reinforcement learning methods and similar environments. We further provide a comprehensive set of ablation analysis to validate the proposed methodology. |
| Tasks | |
| Published | 2019-08-05 |
| URL | https://arxiv.org/abs/1908.01478v1 |
| https://arxiv.org/pdf/1908.01478v1.pdf | |
| PWC | https://paperswithcode.com/paper/construction-of-macro-actions-for-deep |
| Repo | |
| Framework | |
Discriminative Ridge Machine: A Classifier for High-Dimensional Data or Imbalanced Data
| Title | Discriminative Ridge Machine: A Classifier for High-Dimensional Data or Imbalanced Data |
| Authors | Chong Peng, Qiang Cheng |
| Abstract | We introduce a discriminative regression approach to supervised classification in this paper. It estimates a representation model while accounting for discriminativeness between classes, thereby enabling accurate derivation of categorical information. This new type of regression models extends existing models such as ridge, lasso, and group lasso through explicitly incorporating discriminative information. As a special case we focus on a quadratic model that admits a closed-form analytical solution. The corresponding classifier is called discriminative regression machine (DRM). Three iterative algorithms are further established for the DRM to enhance the efficiency and scalability for real applications. Our approach and the algorithms are applicable to general types of data including images, high-dimensional data, and imbalanced data. We compare the DRM with currently state-of-the-art classifiers. Our extensive experimental results show superior performance of the DRM and confirm the effectiveness of the proposed approach. |
| Tasks | |
| Published | 2019-04-16 |
| URL | https://arxiv.org/abs/1904.07496v2 |
| https://arxiv.org/pdf/1904.07496v2.pdf | |
| PWC | https://paperswithcode.com/paper/discriminative-regression-machine-a |
| Repo | |
| Framework | |
AR-Net: A simple Auto-Regressive Neural Network for time-series
| Title | AR-Net: A simple Auto-Regressive Neural Network for time-series |
| Authors | Oskar Triebe, Nikolay Laptev, Ram Rajagopal |
| Abstract | In this paper we present a new framework for time-series modeling that combines the best of traditional statistical models and neural networks. We focus on time-series with long-range dependencies, needed for monitoring fine granularity data (e.g. minutes, seconds, milliseconds), prevalent in operational use-cases. Traditional models, such as auto-regression fitted with least squares (Classic-AR) can model time-series with a concise and interpretable model. When dealing with long-range dependencies, Classic-AR models can become intractably slow to fit for large data. Recently, sequence-to-sequence models, such as Recurrent Neural Networks, which were originally intended for natural language processing, have become popular for time-series. However, they can be overly complex for typical time-series data and lack interpretability. A scalable and interpretable model is needed to bridge the statistical and deep learning-based approaches. As a first step towards this goal, we propose modelling AR-process dynamics using a feed-forward neural network approach, termed AR-Net. We show that AR-Net is as interpretable as Classic-AR but also scales to long-range dependencies. Our results lead to three major conclusions: First, AR-Net learns identical AR-coefficients as Classic-AR, thus being equally interpretable. Second, the computational complexity with respect to the order of the AR process, is linear for AR-Net as compared to a quadratic for Classic-AR. This makes it possible to model long-range dependencies within fine granularity data. Third, by introducing regularization, AR-Net automatically selects and learns sparse AR-coefficients. This eliminates the need to know the exact order of the AR-process and allows to learn sparse weights for a model with long-range dependencies. |
| Tasks | Time Series |
| Published | 2019-11-27 |
| URL | https://arxiv.org/abs/1911.12436v1 |
| https://arxiv.org/pdf/1911.12436v1.pdf | |
| PWC | https://paperswithcode.com/paper/ar-net-a-simple-auto-regressive-neural |
| Repo | |
| Framework | |
Disentangling Options with Hellinger Distance Regularizer
| Title | Disentangling Options with Hellinger Distance Regularizer |
| Authors | Minsung Hyun, Junyoung Choi, Nojun Kwak |
| Abstract | In reinforcement learning (RL), temporal abstraction still remains as an important and unsolved problem. The options framework provided clues to temporal abstraction in the RL, and the option-critic architecture elegantly solved the two problems of finding options and learning RL agents in an end-to-end manner. However, it is necessary to examine whether the options learned through this method play a mutually exclusive role. In this paper, we propose a Hellinger distance regularizer, a method for disentangling options. In addition, we will shed light on various indicators from the statistical point of view to compare with the options learned through the existing option-critic architecture. |
| Tasks | |
| Published | 2019-04-15 |
| URL | http://arxiv.org/abs/1904.06887v1 |
| http://arxiv.org/pdf/1904.06887v1.pdf | |
| PWC | https://paperswithcode.com/paper/disentangling-options-with-hellinger-distance |
| Repo | |
| Framework | |
Boosting Classifiers with Noisy Inference
| Title | Boosting Classifiers with Noisy Inference |
| Authors | Yongjune Kim, Yuval Cassuto, Lav R. Varshney |
| Abstract | We present a principled framework to address resource allocation for realizing boosting algorithms on substrates with communication or computation noise. Boosting classifiers (e.g., AdaBoost) make a final decision via a weighted vote from the outputs of many base classifiers (weak classifiers). Suppose that the base classifiers’ outputs are noisy or communicated over noisy channels; these noisy outputs will degrade the final classification accuracy. We show that this degradation can be effectively reduced by allocating more system resources for more important base classifiers. We formulate resource optimization problems in terms of importance metrics for boosting. Moreover, we show that the optimized noisy boosting classifiers can be more robust than bagging for the noise during inference (test stage). We provide numerical evidence to demonstrate the benefits of our approach. |
| Tasks | |
| Published | 2019-09-10 |
| URL | https://arxiv.org/abs/1909.04766v1 |
| https://arxiv.org/pdf/1909.04766v1.pdf | |
| PWC | https://paperswithcode.com/paper/boosting-classifiers-with-noisy-inference |
| Repo | |
| Framework | |
Toward Standardized Classification of Foveated Displays
| Title | Toward Standardized Classification of Foveated Displays |
| Authors | Josef Spjut, Ben Boudaoud, Jonghyun Kim, Trey Greer, Rachel Albert, Michael Stengel, Kaan Aksit, David Luebke |
| Abstract | Emergent in the field of head mounted display design is a desire to leverage the limitations of the human visual system to reduce the computation, communication, and display workload in power and form-factor constrained systems. Fundamental to this reduced workload is the ability to match display resolution to the acuity of the human visual system, along with a resulting need to follow the gaze of the eye as it moves, a process referred to as foveation. A display that moves its content along with the eye may be called a Foveated Display, though this term is also commonly used to describe displays with non-uniform resolution that attempt to mimic human visual acuity. We therefore recommend a definition for the term Foveated Display that accepts both of these interpretations. Furthermore, we include a simplified model for human visual Acuity Distribution Functions (ADFs) at various levels of visual acuity, across wide fields of view and propose comparison of this ADF with the Resolution Distribution Function of a foveated display for evaluation of its resolution at a particular gaze direction. We also provide a taxonomy to allow the field to meaningfully compare and contrast various aspects of foveated displays in a display and optical technology-agnostic manner. |
| Tasks | Foveation |
| Published | 2019-05-03 |
| URL | https://arxiv.org/abs/1905.06229v1 |
| https://arxiv.org/pdf/1905.06229v1.pdf | |
| PWC | https://paperswithcode.com/paper/190506229 |
| Repo | |
| Framework | |
Exploring the context of recurrent neural network based conversational agents
| Title | Exploring the context of recurrent neural network based conversational agents |
| Authors | Raffaele Piccini, Gerasimos Spanakis |
| Abstract | Conversational agents have begun to rise both in the academic (in terms of research) and commercial (in terms of applications) world. This paper investigates the task of building a non-goal driven conversational agent, using neural network generative models and analyzes how the conversation context is handled. It compares a simpler Encoder-Decoder with a Hierarchical Recurrent Encoder-Decoder architecture, which includes an additional module to model the context of the conversation using previous utterances information. We found that the hierarchical model was able to extract relevant context information and include them in the generation of the output. However, it performed worse (35-40%) than the simple Encoder-Decoder model regarding both grammatically correct output and meaningful response. Despite these results, experiments demonstrate how conversations about similar topics appear close to each other in the context space due to the increased frequency of specific topic-related words, thus leaving promising directions for future research and how the context of a conversation can be exploited. |
| Tasks | |
| Published | 2019-01-31 |
| URL | http://arxiv.org/abs/1901.11462v1 |
| http://arxiv.org/pdf/1901.11462v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-the-context-of-recurrent-neural |
| Repo | |
| Framework | |
Unsupervised Change Detection in Multi-temporal VHR Images Based on Deep Kernel PCA Convolutional Mapping Network
| Title | Unsupervised Change Detection in Multi-temporal VHR Images Based on Deep Kernel PCA Convolutional Mapping Network |
| Authors | Chen Wu, Hongruixuan Chen, Bo Do, Liangpei Zhang |
| Abstract | With the development of Earth observation technology, very-high-resolution (VHR) image has become an important data source of change detection. Nowadays, deep learning methods have achieved conspicuous performance in the change detection of VHR images. Nonetheless, most of the existing change detection models based on deep learning require annotated training samples. In this paper, a novel unsupervised model called kernel principal component analysis (KPCA) convolution is proposed for extracting representative features from multi-temporal VHR images. Based on the KPCA convolution, an unsupervised deep siamese KPCA convolutional mapping network (KPCA-MNet) is designed for binary and multi-class change detection. In the KPCA-MNet, the high-level spatial-spectral feature maps are extracted by a deep siamese network consisting of weight-shared PCA convolution layers. Then, the change information in the feature difference map is mapped into a 2-D polar domain. Finally, the change detection results are generated by threshold segmentation and clustering algorithms. All procedures of KPCA-MNet does not require labeled data. The theoretical analysis and experimental results demonstrate the validity, robustness, and potential of the proposed method in two binary change detection data sets and one multi-class change detection data set. |
| Tasks | |
| Published | 2019-12-18 |
| URL | https://arxiv.org/abs/1912.08628v1 |
| https://arxiv.org/pdf/1912.08628v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-change-detection-in-multi |
| Repo | |
| Framework | |
Convergence Analysis of Gradient-Based Learning with Non-Uniform Learning Rates in Non-Cooperative Multi-Agent Settings
| Title | Convergence Analysis of Gradient-Based Learning with Non-Uniform Learning Rates in Non-Cooperative Multi-Agent Settings |
| Authors | Benjamin Chasnov, Lillian J. Ratliff, Eric Mazumdar, Samuel A. Burden |
| Abstract | Considering a class of gradient-based multi-agent learning algorithms in non-cooperative settings, we provide local convergence guarantees to a neighborhood of a stable local Nash equilibrium. In particular, we consider continuous games where agents learn in (i) deterministic settings with oracle access to their gradient and (ii) stochastic settings with an unbiased estimator of their gradient. Utilizing the minimum and maximum singular values of the game Jacobian, we provide finite-time convergence guarantees in the deterministic case. On the other hand, in the stochastic case, we provide concentration bounds guaranteeing that with high probability agents will converge to a neighborhood of a stable local Nash equilibrium in finite time. Different than other works in this vein, we also study the effects of non-uniform learning rates on the learning dynamics and convergence rates. We find that much like preconditioning in optimization, non-uniform learning rates cause a distortion in the vector field which can, in turn, change the rate of convergence and the shape of the region of attraction. The analysis is supported by numerical examples that illustrate different aspects of the theory. We conclude with discussion of the results and open questions. |
| Tasks | |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1906.00731v1 |
| https://arxiv.org/pdf/1906.00731v1.pdf | |
| PWC | https://paperswithcode.com/paper/190600731 |
| Repo | |
| Framework | |
A Fine-Grained Variant of the Hierarchy of Lasserre
| Title | A Fine-Grained Variant of the Hierarchy of Lasserre |
| Authors | Wann-Jiun Ma, Jakub Marecek, Martin Mevissen |
| Abstract | There has been much recent interest in hierarchies of progressively stronger convexifications of polynomial optimisation problems (POP). These often converge to the global optimum of the POP, asymptotically, but prove challenging to solve beyond the first level in the hierarchy for modest instances. We present a finer-grained variant of the Lasserre hierarchy, together with first-order methods for solving the convexifications, which allow for efficient warm-starting with solutions from lower levels in the hierarchy. |
| Tasks | |
| Published | 2019-06-23 |
| URL | https://arxiv.org/abs/1906.09586v1 |
| https://arxiv.org/pdf/1906.09586v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-fine-grained-variant-of-the-hierarchy-of |
| Repo | |
| Framework | |
Photosequencing of Motion Blur using Short and Long Exposures
| Title | Photosequencing of Motion Blur using Short and Long Exposures |
| Authors | Vijay Rengarajan, Shuo Zhao, Ruiwen Zhen, John Glotzbach, Hamid Sheikh, Aswin C. Sankaranarayanan |
| Abstract | Photosequencing aims to transform a motion blurred image to a sequence of sharp images. This problem is challenging due to the inherent ambiguities in temporal ordering as well as the recovery of lost spatial textures due to blur. Adopting a computational photography approach, we propose to capture two short exposure images, along with the original blurred long exposure image to aid in the aforementioned challenges. Post-capture, we recover the sharp photosequence using a novel blur decomposition strategy that recursively splits the long exposure image into smaller exposure intervals. We validate the approach by capturing a variety of scenes with interesting motions using machine vision cameras programmed to capture short and long exposure sequences. Our experimental results show that the proposed method resolves both fast and fine motions better than prior works. |
| Tasks | |
| Published | 2019-12-11 |
| URL | https://arxiv.org/abs/1912.06102v1 |
| https://arxiv.org/pdf/1912.06102v1.pdf | |
| PWC | https://paperswithcode.com/paper/photosequencing-of-motion-blur-using-short |
| Repo | |
| Framework | |
Online Antenna Tuning in Heterogeneous Cellular Networks with Deep Reinforcement Learning
| Title | Online Antenna Tuning in Heterogeneous Cellular Networks with Deep Reinforcement Learning |
| Authors | Eren Balevi, Jeffrey G. Andrews |
| Abstract | We aim to jointly optimize antenna tilt angle, and vertical and horizontal half-power beamwidths of the macrocells in a heterogeneous cellular network (HetNet). The interactions between the cells, most notably due to their coupled interference render this optimization prohibitively complex. Utilizing a single agent reinforcement learning (RL) algorithm for this optimization becomes quite suboptimum despite its scalability, whereas multi-agent RL algorithms yield better solutions at the expense of scalability. Hence, we propose a compromise algorithm between these two. Specifically, a multi-agent mean field RL algorithm is first utilized in the offline phase so as to transfer information as features for the second (online) phase single agent RL algorithm, which employs a deep neural network to learn users locations. This two-step approach is a practical solution for real deployments, which should automatically adapt to environmental changes in the network. Our results illustrate that the proposed algorithm approaches the performance of the multi-agent RL, which requires millions of trials, with hundreds of online trials, assuming relatively low environmental dynamics, and performs much better than a single agent RL. Furthermore, the proposed algorithm is compact and implementable, and empirically appears to provide a performance guarantee regardless of the amount of environmental dynamics. |
| Tasks | Q-Learning |
| Published | 2019-03-15 |
| URL | https://arxiv.org/abs/1903.06787v2 |
| https://arxiv.org/pdf/1903.06787v2.pdf | |
| PWC | https://paperswithcode.com/paper/online-antenna-tuning-in-heterogeneous |
| Repo | |
| Framework | |
