January 31, 2020

3197 words 16 mins read

Paper Group ANR 24

Accurate Face Detection for High Performance. RKHSMetaMod: An R package to estimate the Hoeffding decomposition of an unknown function by solving RKHS ridge group sparse optimization problem. Local Context Normalization: Revisiting Local Normalization. Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Network …

Accurate Face Detection for High Performance


Title	Accurate Face Detection for High Performance
Authors	Faen Zhang, Xinyu Fan, Guo Ai, Jianfei Song, Yongqiang Qin, Jiahong Wu
Abstract	Face detection has witnessed significant progress due to the advances of deep convolutional neural networks (CNNs). Its central issue in recent years is how to improve the detection performance of tiny faces. To this end, many recent works propose some specific strategies, redesign the architecture and introduce new loss functions for tiny object detection. In this report, we start from the popular one-stage RetinaNet approach and apply some recent tricks to obtain a high performance face detector. Specifically, we apply the Intersection over Union (IoU) loss function for regression, employ the two-step classification and regression for detection, revisit the data augmentation based on data-anchor-sampling for training, utilize the max-out operation for classification and use the multi-scale testing strategy for inference. As a consequence, the proposed face detection method achieves state-of-the-art performance on the most popular and challenging face detection benchmark WIDER FACE dataset.
Tasks	Data Augmentation, Face Detection, Object Detection
Published	2019-05-05
URL	https://arxiv.org/abs/1905.01585v3
PDF	https://arxiv.org/pdf/1905.01585v3.pdf
PWC	https://paperswithcode.com/paper/accurate-face-detection-for-high-performance
Repo
Framework

RKHSMetaMod: An R package to estimate the Hoeffding decomposition of an unknown function by solving RKHS ridge group sparse optimization problem


Title	RKHSMetaMod: An R package to estimate the Hoeffding decomposition of an unknown function by solving RKHS ridge group sparse optimization problem
Authors	Halaleh Kamari, Sylvie Huet, Marie-Luce Taupin
Abstract	In the context of the Gaussian regression model, the package RKHSMetaMod allows estimating a meta-model by solving the ridge group sparse optimization problem based on the Reproducing Kernel Hilbert Spaces (RKHS). The obtained estimator is an additive model that satisfies the properties of the Hoeffding decomposition, and its terms estimate the terms in the Hoeffding decomposition of the unknown regression function. The estimators of the Sobol indices are deduced from the estimated meta-model. This package provides an interface from the R statistical computing environment to the C++ libraries Eigen and GSL. In order to speed up the execution time, almost all of the functions of the RKHSMetaMod package are written using the efficient C++ libraries through RcppEigen and RcppGSL packages. These functions are then interfaced in the R environment in order to propose a user-friendly package.
Tasks
Published	2019-05-31
URL	https://arxiv.org/abs/1905.13695v2
PDF	https://arxiv.org/pdf/1905.13695v2.pdf
PWC	https://paperswithcode.com/paper/rkhsmetamod-an-r-package-to-estimate-the
Repo
Framework

Local Context Normalization: Revisiting Local Normalization


Title	Local Context Normalization: Revisiting Local Normalization
Authors	Anthony Ortiz, Caleb Robinson, Dan Morris, Olac Fuentes, Christopher Kiekintveld, Mahmudulla Hassan, Nebojsa Jojic
Abstract	Normalization layers have been shown to improve convergence in deep neural networks. In many vision applications the local spatial context of the features is important, but most common normalization schemes includingGroup Normalization (GN), Instance Normalization (IN), and Layer Normalization (LN) normalize over the entire spatial dimension of a feature. This can wash out important signals and degrade performance. For example, in applications that use satellite imagery, input images can be arbitrarily large; consequently, it is nonsensical to normalize over the entire area. Positional Normalization (PN), on the other hand, only normalizes over a single spatial position at a time. A natural compromise is to normalize features by local context, while also taking into account group level information. In this paper, we propose Local Context Normalization (LCN): a normalization layer where every feature is normalized based on a window around it and the filters in its group. We propose an algorithmic solution to make LCN efficient for arbitrary window sizes, even if every point in the image has a unique window. LCN outperforms its Batch Normalization (BN), GN, IN, and LN counterparts for object detection, semantic segmentation, and instance segmentation applications in several benchmark datasets, while keeping performance independent of the batch size and facilitating transfer learning.
Tasks	Instance Segmentation, Object Detection, Semantic Segmentation, Transfer Learning
Published	2019-12-12
URL	https://arxiv.org/abs/1912.05845v2
PDF	https://arxiv.org/pdf/1912.05845v2.pdf
PWC	https://paperswithcode.com/paper/local-context-normalization-revisiting-local
Repo
Framework

Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks


Title	Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks
Authors	Yuan Cao, Quanquan Gu
Abstract	Empirical studies show that gradient-based methods can learn deep neural networks (DNNs) with very good generalization performance in the over-parameterization regime, where DNNs can easily fit a random labeling of the training data. Very recently, a line of work explains in theory that with over-parameterization and proper random initialization, gradient-based methods can find the global minima of the training loss for DNNs. However, existing generalization error bounds are unable to explain the good generalization performance of over-parameterized DNNs. The major limitation of most existing generalization bounds is that they are based on uniform convergence and are independent of the training algorithm. In this work, we derive an algorithm-dependent generalization error bound for deep ReLU networks, and show that under certain assumptions on the data distribution, gradient descent (GD) with proper random initialization is able to train a sufficiently over-parameterized DNN to achieve arbitrarily small generalization error. Our work sheds light on explaining the good generalization performance of over-parameterized deep neural networks.
Tasks
Published	2019-02-04
URL	https://arxiv.org/abs/1902.01384v4
PDF	https://arxiv.org/pdf/1902.01384v4.pdf
PWC	https://paperswithcode.com/paper/a-generalization-theory-of-gradient-descent
Repo
Framework

Graph Spectral Characterization of Brain Cortical Morphology


Title	Graph Spectral Characterization of Brain Cortical Morphology
Authors	Sevil Maghsadhagh, Anders Eklund, Hamid Behjat
Abstract	The human brain cortical layer has a convoluted morphology that is unique to each individual. Characterization of the cortical morphology is necessary in longitudinal studies of structural brain change, as well as in discriminating individuals in health and disease. A method for encoding the cortical morphology in the form of a graph is presented. The design of graphs that encode the global cerebral hemisphere cortices as well as localized cortical regions is proposed. Spectral metrics derived from these graphs are then studied and proposed as descriptors of cortical morphology. As proof-of-concept of their applicability in characterizing cortical morphology, the metrics are studied in the context of hemispheric asymmetry as well as gender dependent discrimination of cortical morphology.
Tasks
Published	2019-02-19
URL	http://arxiv.org/abs/1902.07283v1
PDF	http://arxiv.org/pdf/1902.07283v1.pdf
PWC	https://paperswithcode.com/paper/graph-spectral-characterization-of-brain
Repo
Framework

Ensemble of Convolutional Neural Networks Trained with Different Activation Functions


Title	Ensemble of Convolutional Neural Networks Trained with Different Activation Functions
Authors	Gianluca Maguolo, Loris Nanni, Stefano Ghidoni
Abstract	Activation functions play a vital role in the training of Convolutional Neural Networks. For this reason, to develop efficient and performing functions is a crucial problem in the deep learning community. Key to these approaches is to permit a reliable parameter learning, avoiding vanishing gradient problems. The goal of this work is to propose an ensemble of Convolutional Neural Networks trained using several different activation functions. Moreover, a novel activation function is here proposed for the first time. Our aim is to improve the performance of Convolutional Neural Networks in small/medium size biomedical datasets. Our results clearly show that the proposed ensemble outperforms Convolutional Neural Networks trained with standard ReLU as activation function. The proposed ensemble outperforms with a p-value of 0.01 each tested stand-alone activation function; for reliable performance comparison we have tested our approach in more than 10 datasets, using two well-known Convolutional Neural Network: Vgg16 and ResNet50. MATLAB code used here will be available at https://github.com/LorisNanni.
Tasks
Published	2019-05-07
URL	https://arxiv.org/abs/1905.02473v4
PDF	https://arxiv.org/pdf/1905.02473v4.pdf
PWC	https://paperswithcode.com/paper/ensemble-of-convolutional-neural-networks
Repo
Framework


Title	See More Than Once – Kernel-Sharing Atrous Convolution for Semantic Segmentation
Authors	Ye Huang, Qingqing Wang, Wenjing Jia, Xiangjian He
Abstract	The state-of-the-art semantic segmentation solutions usually leverage different receptive fields via multiple parallel branches to handle objects with different sizes. However, employing separate kernels for individual branches degrades the generalization and representation abilities of the network, and the number of parameters increases linearly in the number of branches. To tackle this problem, we propose a novel network structure namely Kernel-Sharing Atrous Convolution (KSAC), where branches of different receptive fields share the same kernel, i.e., let a single kernel see the input feature maps more than once with different receptive fields, to facilitate communication among branches and perform feature augmentation inside the network. Experiments conducted on the benchmark PASCAL VOC 2012 dataset show that the proposed sharing strategy can not only boost a network s generalization and representation abilities but also reduce the model complexity significantly. Specifically, on the validation set, whe compared with DeepLabV3+ equipped with MobileNetv2 backbone, 33% of parameters are reduced together with an mIOU improvement of 0.6%. When Xception is used as the backbone, the mIOU is elevated from 83.34% to 85.96% with about 10M parameters saved. In addition, different from the widely used ASPP structure, our proposed KSAC is able to further improve the mIOU by taking benefit of wider context with larger atrous rates. Finally, our KSAC achieves mIOUs of 88.1% and 45.47% on the PASCAL VOC 2012 test set and ADE20K dataset, respectively. Our full code will be released on the Github.
Tasks	Semantic Segmentation
Published	2019-08-26
URL	https://arxiv.org/abs/1908.09443v4
PDF	https://arxiv.org/pdf/1908.09443v4.pdf
PWC	https://paperswithcode.com/paper/see-more-than-once-kernel-sharing-atrous
Repo
Framework

Nonlinear Approximation via Compositions


Title	Nonlinear Approximation via Compositions
Authors	Zuowei Shen, Haizhao Yang, Shijun Zhang
Abstract	We study the approximation efficiency of function compositions in nonlinear approximation, especially the case when compositions are implemented using multi-layer feed-forward neural networks (FNNs) with ReLU activation functions. The central question of interest is what are the advantages of function compositions in generating dictionaries and what is the optimal implementation of function compositions via ReLU FNNs, especially in modern computing architecture. This question is answered by studying the $N$-term approximation rate, which is the decrease in error versus the number of computational nodes (neurons) in the approximant, together with parallel efficiency for the first time. First, for an arbitrary function $f$ on $[0,1]$, regardless of its smoothness and even the continuity, if $f$ can be approximated via nonlinear approximation using one-hidden-layer ReLU FNNs with an approximation rate $O(N^{-\eta})$, we quantitatively show that dictionaries with function compositions via deep ReLU FNNs can improve the approximation rate to $O(N^{-2\eta})$. Second, for H{"o}lder continuous functions of order $\alpha$ with a uniform Lipchitz constant $\omega$ on a $d$-dimensional cube, we show that the $N$-term approximation via ReLU FNNs with two or three function compositions can achieve an approximation rate $O( N^{-2\alpha/d})$. The approximation rate can be improved to $O(L^{-2\alpha/d})$ by composing $L$ times, if $N$ is fixed and sufficiently large; but further compositions cannot achieve the approximation rate $O(N^{-\alpha L/d})$. Finally, considering the computational efficiency per training iteration in parallel computing, FNNs with $O(1)$ hidden layers are an optimal choice for approximating H{"o}lder continuous functions if computing resources are enough.
Tasks
Published	2019-02-26
URL	https://arxiv.org/abs/1902.10170v4
PDF	https://arxiv.org/pdf/1902.10170v4.pdf
PWC	https://paperswithcode.com/paper/nonlinear-approximation-via-compositions
Repo
Framework

iProStruct2D: Identifying protein structural classes by deep learning via 2D representations


Title	iProStruct2D: Identifying protein structural classes by deep learning via 2D representations
Authors	Loris Nanni, Alessandra Lumini, Federica Pasquali, Sheryl Brahnam
Abstract	In this paper we address the problem of protein classification starting from a multi-view 2D representation of proteins. From each 3D protein structure, a large set of 2D projections is generated using the protein visualization software Jmol. This set of multi-view 2D representations includes 13 different types of protein visualizations that emphasize specific properties of protein structure (e.g., a backbone visualization that displays the backbone structure of the protein as a trace of the C{\alpha} atom). Each type of representation is used to train a different Convolutional Neural Network (CNN), and the fusion of these CNNs is shown to be able to exploit the diversity of different types of representations to improve classification performance. In addition, several multi-view projections are obtained by uniformly rotating the protein structure around its central X, Y, and Z viewing axes to produce 125 images. This approach can be considered a data augmentation method for improving the performance of the classifier and can be used in both the training and the testing phases. Experimental evaluation of the proposed approach on two datasets demonstrates the strength of the proposed method with respect to the other state-of-the-art approaches. The MATLAB code used in this paper is available at https://github.com/LorisNanni.
Tasks	Data Augmentation
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04407v1
PDF	https://arxiv.org/pdf/1906.04407v1.pdf
PWC	https://paperswithcode.com/paper/iprostruct2d-identifying-protein-structural
Repo
Framework

Lecturer Performance System Using Neural Network with Particle Swarm Optimization


Title	Lecturer Performance System Using Neural Network with Particle Swarm Optimization
Authors	Tarik A. Rashid, Hawraz A. Ahmad
Abstract	The field of analyzing performance is very important and sensitive in particular when it is related to the performance of lecturers in academic institutions. Locating the weak points of lecturers through a system that provides an early warning to notify or reward the lecturers with warned or punished notices will help them to improve their weaknesses, leads to a better quality in the institutions. The current system has major issues in the higher education at Salahaddin University-Erbil (SUE) in Kurdistan-Iraq. These issues are: first, the assessment of lecturers’ activities is conducted traditionally via the Quality Assurance Teams at different departments and colleges at the university, second, the outcomes in some cases of lecturers’ performance provoke a low level of acceptance among lectures, as these cases are reflected and viewed by some academic communities as unfair cases, and finally, the current system is not accurate and vigorous. In this paper, Particle Swarm Optimization Neural Network is used to assess performance of lecturers in more fruitful way and also to enhance the accuracy of recognition system. Different real and novel data sets are collected from SUE. The prepared datasets preprocessed and important features are then fed as input source to the training and testing phases. Particle Swarm Optimization is used to find the best weights and biases in the training phase of the neural network. The best accuracy rate obtained in the test phase is 98.28 %.
Tasks
Published	2019-04-08
URL	http://arxiv.org/abs/1904.04377v1
PDF	http://arxiv.org/pdf/1904.04377v1.pdf
PWC	https://paperswithcode.com/paper/lecturer-performance-system-using-neural
Repo
Framework

Adversarial Transformations for Semi-Supervised Learning


Title	Adversarial Transformations for Semi-Supervised Learning
Authors	Teppei Suzuki, Ikuro Sato
Abstract	We propose a Regularization framework based on Adversarial Transformations (RAT) for semi-supervised learning. RAT is designed to enhance robustness of the output distribution of class prediction for a given data against input perturbation. RAT is an extension of Virtual Adversarial Training (VAT) in such a way that RAT adversarialy transforms data along the underlying data distribution by a rich set of data transformation functions that leave class label invariant, whereas VAT simply produces adversarial additive noises. In addition, we verified that a technique of gradually increasing of perturbation region further improve the robustness. In experiments, we show that RAT significantly improves classification performance on CIFAR-10 and SVHN compared to existing regularization methods under standard semi-supervised image classification settings.
Tasks	Image Classification, Semi-Supervised Image Classification
Published	2019-11-13
URL	https://arxiv.org/abs/1911.06181v2
PDF	https://arxiv.org/pdf/1911.06181v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-transformations-for-semi
Repo
Framework

G2R Bound: A Generalization Bound for Supervised Learning from GAN-Synthetic Data


Title	G2R Bound: A Generalization Bound for Supervised Learning from GAN-Synthetic Data
Authors	Fu-Chieh Chang, Hao-Jen Wang, Chun-Nan Chou, Edward Y. Chang
Abstract	Performing supervised learning from the data synthesized by using Generative Adversarial Networks (GANs), dubbed GAN-synthetic data, has two important applications. First, GANs may generate more labeled training data, which may help improve classification accuracy. Second, in scenarios where real data cannot be released outside certain premises for privacy and/or security reasons, using GAN- synthetic data to conduct training is a plausible alternative. This paper proposes a generalization bound to guarantee the generalization capability of a classifier learning from GAN-synthetic data. This generalization bound helps developers gauge the generalization gap between learning from synthetic data and testing on real data, and can therefore provide the clues to improve the generalization capability.
Tasks
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12313v1
PDF	https://arxiv.org/pdf/1905.12313v1.pdf
PWC	https://paperswithcode.com/paper/g2r-bound-a-generalization-bound-for
Repo
Framework

Boosting Local Causal Discovery in High-Dimensional Expression Data


Title	Boosting Local Causal Discovery in High-Dimensional Expression Data
Authors	Philip Versteeg, Joris M. Mooij
Abstract	We study the performance of Local Causal Discovery (LCD), a simple and efficient constraint-based method for causal discovery, in predicting causal effects in large-scale gene expression data. We construct practical estimators specific to the high-dimensional regime. Inspired by the ICP algorithm, we use an optional preselection method and two different statistical tests. Empirically, the resulting LCD estimator is seen to closely approach the accuracy of ICP, the state-of-the-art method, while it is algorithmically simpler and computationally more efficient.
Tasks	Causal Discovery
Published	2019-10-06
URL	https://arxiv.org/abs/1910.02505v2
PDF	https://arxiv.org/pdf/1910.02505v2.pdf
PWC	https://paperswithcode.com/paper/boosting-local-causal-discovery-in-high
Repo
Framework

Expansion of Cyber Attack Data From Unbalanced Datasets Using Generative Techniques


Title	Expansion of Cyber Attack Data From Unbalanced Datasets Using Generative Techniques
Authors	Ibrahim Yilmaz, Rahat Masum
Abstract	Machine learning techniques help to understand patterns of a dataset to create a defense mechanism against cyber attacks. However, it is difficult to construct a theoretical model due to the imbalances in the dataset for discriminating attacks from the overall dataset. Multilayer Perceptron (MLP) technique will provide improvement in accuracy and increase the performance of detecting the attack and benign data from a balanced dataset. We have worked on the UGR’16 dataset publicly available for this work. Data wrangling has been done due to prepare test set from in the original set. We fed the neural network classifier larger input to the neural network in an increasing manner (i.e. 10000, 50000, 1 million) to see the distribution of features over the accuracy. We have implemented a GAN model that can produce samples of different attack labels (e.g. blacklist, anomaly spam, ssh scan). We have been able to generate as many samples as necessary based on the data sample we have taken from the UGR’16. We have tested the accuracy of our model with the imbalance dataset initially and then with the increasing the attack samples and found improvement of classification performance for the latter.
Tasks
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04549v1
PDF	https://arxiv.org/pdf/1912.04549v1.pdf
PWC	https://paperswithcode.com/paper/expansion-of-cyber-attack-data-from
Repo
Framework

Combining Benefits from Trajectory Optimization and Deep Reinforcement Learning


Title	Combining Benefits from Trajectory Optimization and Deep Reinforcement Learning
Authors	Guillaume Bellegarda, Katie Byl
Abstract	Recent breakthroughs both in reinforcement learning and trajectory optimization have made significant advances towards real world robotic system deployment. Reinforcement learning (RL) can be applied to many problems without needing any modeling or intuition about the system, at the cost of high sample complexity and the inability to prove any metrics about the learned policies. Trajectory optimization (TO) on the other hand allows for stability and robustness analyses on generated motions and trajectories, but is only as good as the often over-simplified derived model, and may have prohibitively expensive computation times for real-time control. This paper seeks to combine the benefits from these two areas while mitigating their drawbacks by (1) decreasing RL sample complexity by using existing knowledge of the problem with optimal control, and (2) providing an upper bound estimate on the time-to-arrival of the combined learned-optimized policy, allowing online policy deployment at any point in the training process by using the TO as a worst-case scenario action. This method is evaluated for a car model, with applicability to any mobile robotic system. A video showing policy execution comparisons can be found at https://youtu.be/mv2xw83NyWU .
Tasks
Published	2019-10-21
URL	https://arxiv.org/abs/1910.09667v1
PDF	https://arxiv.org/pdf/1910.09667v1.pdf
PWC	https://paperswithcode.com/paper/combining-benefits-from-trajectory
Repo
Framework