Paper Group ANR 153
A Statistical Approach to Increase Classification Accuracy in Supervised Learning Algorithms. EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks. Sparse Deep Nonnegative Matrix Factorization. Combinational neural network using Gabor filters for the classification of handwritten digits. Generate Identity-Preserving Face …
A Statistical Approach to Increase Classification Accuracy in Supervised Learning Algorithms
| Title | A Statistical Approach to Increase Classification Accuracy in Supervised Learning Algorithms |
| Authors | Gustavo A Valencia-Zapata, Daniel Mejia, Gerhard Klimeck, Michael Zentner, Okan Ersoy |
| Abstract | Probabilistic mixture models have been widely used for different machine learning and pattern recognition tasks such as clustering, dimensionality reduction, and classification. In this paper, we focus on trying to solve the most common challenges related to supervised learning algorithms by using mixture probability distribution functions. With this modeling strategy, we identify sub-labels and generate synthetic data in order to reach better classification accuracy. It means we focus on increasing the training data synthetically to increase the classification accuracy. |
| Tasks | Dimensionality Reduction |
| Published | 2017-09-05 |
| URL | http://arxiv.org/abs/1709.01439v1 |
| http://arxiv.org/pdf/1709.01439v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-statistical-approach-to-increase |
| Repo | |
| Framework | |
EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks
| Title | EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks |
| Authors | Xuanyi Dong, Guoliang Kang, Kun Zhan, Yi Yang |
| Abstract | For most state-of-the-art architectures, Rectified Linear Unit (ReLU) becomes a standard component accompanied with each layer. Although ReLU can ease the network training to an extent, the character of blocking negative values may suppress the propagation of useful information and leads to the difficulty of optimizing very deep Convolutional Neural Networks (CNNs). Moreover, stacking layers with nonlinear activations is hard to approximate the intrinsic linear transformations between feature representations. In this paper, we investigate the effect of erasing ReLUs of certain layers and apply it to various representative architectures following deterministic rules. It can ease the optimization and improve the generalization performance for very deep CNN models. We find two key factors being essential to the performance improvement: 1) the location where ReLU should be erased inside the basic module; 2) the proportion of basic modules to erase ReLU; We show that erasing the last ReLU layer of all basic modules in a network usually yields improved performance. In experiments, our approach successfully improves the performance of various representative architectures, and we report the improved results on SVHN, CIFAR-10/100, and ImageNet. Moreover, we achieve competitive single-model performance on CIFAR-100 with 16.53% error rate compared to state-of-the-art. |
| Tasks | Image Classification |
| Published | 2017-09-22 |
| URL | http://arxiv.org/abs/1709.07634v2 |
| http://arxiv.org/pdf/1709.07634v2.pdf | |
| PWC | https://paperswithcode.com/paper/eraserelu-a-simple-way-to-ease-the-training |
| Repo | |
| Framework | |
Sparse Deep Nonnegative Matrix Factorization
| Title | Sparse Deep Nonnegative Matrix Factorization |
| Authors | Zhenxing Guo, Shihua Zhang |
| Abstract | Nonnegative matrix factorization is a powerful technique to realize dimension reduction and pattern recognition through single-layer data representation learning. Deep learning, however, with its carefully designed hierarchical structure, is able to combine hidden features to form more representative features for pattern recognition. In this paper, we proposed sparse deep nonnegative matrix factorization models to analyze complex data for more accurate classification and better feature interpretation. Such models are designed to learn localized features or generate more discriminative representations for samples in distinct classes by imposing $L_1$-norm penalty on the columns of certain factors. By extending one-layer model into multi-layer one with sparsity, we provided a hierarchical way to analyze big data and extract hidden features intuitively due to nonnegativity. We adopted the Nesterov’s accelerated gradient algorithm to accelerate the computing process with the convergence rate of $O(1/k^2)$ after $k$ steps iteration. We also analyzed the computing complexity of our framework to demonstrate their efficiency. To improve the performance of dealing with linearly inseparable data, we also considered to incorporate popular nonlinear functions into this framework and explored their performance. We applied our models onto two benchmarking image datasets, demonstrating our models can achieve competitive or better classification performance and produce intuitive interpretations compared with the typical NMF and competing multi-layer models. |
| Tasks | Dimensionality Reduction, Representation Learning |
| Published | 2017-07-28 |
| URL | http://arxiv.org/abs/1707.09316v1 |
| http://arxiv.org/pdf/1707.09316v1.pdf | |
| PWC | https://paperswithcode.com/paper/sparse-deep-nonnegative-matrix-factorization |
| Repo | |
| Framework | |
Combinational neural network using Gabor filters for the classification of handwritten digits
| Title | Combinational neural network using Gabor filters for the classification of handwritten digits |
| Authors | N. Joshi |
| Abstract | A classification algorithm that combines the components of k-nearest neighbours and multilayer neural networks has been designed and tested. With this method the computational time required for training the dataset has been reduced substancially. Gabor filters were used for the feature extraction to ensure a better performance. This algorithm is tested with MNIST dataset and it will be integrated as a module in the object recognition software which is currently under development. |
| Tasks | Object Recognition |
| Published | 2017-09-18 |
| URL | http://arxiv.org/abs/1709.05867v1 |
| http://arxiv.org/pdf/1709.05867v1.pdf | |
| PWC | https://paperswithcode.com/paper/combinational-neural-network-using-gabor |
| Repo | |
| Framework | |
Generate Identity-Preserving Faces by Generative Adversarial Networks
| Title | Generate Identity-Preserving Faces by Generative Adversarial Networks |
| Authors | Zhigang Li, Yupin Luo |
| Abstract | Generating identity-preserving faces aims to generate various face images keeping the same identity given a target face image. Although considerable generative models have been developed in recent years, it is still challenging to simultaneously acquire high quality of facial images and preserve the identity. Here we propose a compelling method using generative adversarial networks (GAN). Concretely, we leverage the generator of trained GAN to generate plausible faces and FaceNet as an identity-similarity discriminator to ensure the identity. Experimental results show that our method is qualified to generate both plausible and identity-preserving faces with high quality. In addition, our method provides a universal framework which can be realized in various ways by combining different face generators and identity-similarity discriminator. |
| Tasks | |
| Published | 2017-06-10 |
| URL | http://arxiv.org/abs/1706.03227v2 |
| http://arxiv.org/pdf/1706.03227v2.pdf | |
| PWC | https://paperswithcode.com/paper/generate-identity-preserving-faces-by |
| Repo | |
| Framework | |
Optimal Categorical Attribute Transformation for Granularity Change in Relational Databases for Binary Decision Problems in Educational Data Mining
| Title | Optimal Categorical Attribute Transformation for Granularity Change in Relational Databases for Binary Decision Problems in Educational Data Mining |
| Authors | Paulo J. L. Adeodato, Fábio C. Pereira, Rosalvo F. Oliveira Neto |
| Abstract | This paper presents an approach for transforming data granularity in hierarchical databases for binary decision problems by applying regression to categorical attributes at the lower grain levels. Attributes from a lower hierarchy entity in the relational database have their information content optimized through regression on the categories histogram trained on a small exclusive labelled sample, instead of the usual mode category of the distribution. The paper validates the approach on a binary decision task for assessing the quality of secondary schools focusing on how logistic regression transforms the students and teachers attributes into school attributes. Experiments were carried out on Brazilian schools public datasets via 10-fold cross-validation comparison of the ranking score produced also by logistic regression. The proposed approach achieved higher performance than the usual distribution mode transformation and equal to the expert weighing approach measured by the maximum Kolmogorov-Smirnov distance and the area under the ROC curve at 0.01 significance level. |
| Tasks | |
| Published | 2017-02-28 |
| URL | http://arxiv.org/abs/1702.08745v1 |
| http://arxiv.org/pdf/1702.08745v1.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-categorical-attribute-transformation |
| Repo | |
| Framework | |
MixedPeds: Pedestrian Detection in Unannotated Videos using Synthetically Generated Human-agents for Training
| Title | MixedPeds: Pedestrian Detection in Unannotated Videos using Synthetically Generated Human-agents for Training |
| Authors | Ernest C. Cheung, Tsan Kwong Wong, Aniket Bera, Dinesh Manocha |
| Abstract | We present a new method for training pedestrian detectors on an unannotated set of images. We produce a mixed reality dataset that is composed of real-world background images and synthetically generated static human-agents. Our approach is general, robust, and makes no other assumptions about the unannotated dataset regarding the number or location of pedestrians. We automatically extract from the dataset: i) the vanishing point to calibrate the virtual camera, and ii) the pedestrians’ scales to generate a Spawn Probability Map, which is a novel concept that guides our algorithm to place the pedestrians at appropriate locations. After putting synthetic human-agents in the unannotated images, we use these augmented images to train a Pedestrian Detector, with the annotations generated along with the synthetic agents. We conducted our experiments using Faster R-CNN by comparing the detection results on the unannotated dataset performed by the detector trained using our approach and detectors trained with other manually labeled datasets. We showed that our approach improves the average precision by 5-13% over these detectors. |
| Tasks | Pedestrian Detection |
| Published | 2017-07-28 |
| URL | http://arxiv.org/abs/1707.09100v2 |
| http://arxiv.org/pdf/1707.09100v2.pdf | |
| PWC | https://paperswithcode.com/paper/mixedpeds-pedestrian-detection-in-unannotated |
| Repo | |
| Framework | |
The destiny of constant structure discrete time closed semantic systems
| Title | The destiny of constant structure discrete time closed semantic systems |
| Authors | Evgeny Ivanko |
| Abstract | Constant structure closed semantic systems are the systems each element of which receives its definition through the correspondent unchangeable set of other elements of the system. Discrete time means here that the definitions of the elements change iteratively and simultaneously based on the “neighbor portraits” from the previous iteration. I prove that the iterative redefinition process in such class of systems will quickly degenerate into a series of pairwise isomorphic states and discuss some directions of further research. |
| Tasks | |
| Published | 2017-11-19 |
| URL | http://arxiv.org/abs/1711.07071v1 |
| http://arxiv.org/pdf/1711.07071v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-destiny-of-constant-structure-discrete |
| Repo | |
| Framework | |
From Deep to Shallow: Transformations of Deep Rectifier Networks
| Title | From Deep to Shallow: Transformations of Deep Rectifier Networks |
| Authors | Senjian An, Farid Boussaid, Mohammed Bennamoun, Jiankun Hu |
| Abstract | In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a single hidden layer. The transformations of both deep rectifier nets and deep residual nets are conducted to demonstrate the advantages of the residual nets over the conventional neural nets and the advantages of the deep neural nets over the shallow neural nets. In summary, for two rectifier nets with different depths but with same total number of hidden units, the corresponding single hidden layer representation of the deeper net is much more complex than the corresponding single hidden representation of the shallower net. Similarly, for a residual net and a conventional rectifier net with the same structure except for the skip connections in the residual net, the corresponding single hidden layer representation of the residual net is much more complex than the corresponding single hidden layer representation of the conventional net. |
| Tasks | |
| Published | 2017-03-30 |
| URL | http://arxiv.org/abs/1703.10355v1 |
| http://arxiv.org/pdf/1703.10355v1.pdf | |
| PWC | https://paperswithcode.com/paper/from-deep-to-shallow-transformations-of-deep |
| Repo | |
| Framework | |
3D Binary Signatures
| Title | 3D Binary Signatures |
| Authors | Siddharth Srivastava, Brejesh Lall |
| Abstract | In this paper, we propose a novel binary descriptor for 3D point clouds. The proposed descriptor termed as 3D Binary Signature (3DBS) is motivated from the matching efficiency of the binary descriptors for 2D images. 3DBS describes keypoints from point clouds with a binary vector resulting in extremely fast matching. The method uses keypoints from standard keypoint detectors. The descriptor is built by constructing a Local Reference Frame and aligning a local surface patch accordingly. The local surface patch constitutes of identifying nearest neighbours based upon an angular constraint among them. The points are ordered with respect to the distance from the keypoints. The normals of the ordered pairs of these keypoints are projected on the axes and the relative magnitude is used to assign a binary digit. The vector thus constituted is used as a signature for representing the keypoints. The matching is done by using hamming distance. We show that 3DBS outperforms state of the art descriptors on various evaluation metrics. |
| Tasks | |
| Published | 2017-08-26 |
| URL | http://arxiv.org/abs/1708.07937v1 |
| http://arxiv.org/pdf/1708.07937v1.pdf | |
| PWC | https://paperswithcode.com/paper/3d-binary-signatures |
| Repo | |
| Framework | |
Integral Policy Iterations for Reinforcement Learning Problems in Continuous Time and Space
| Title | Integral Policy Iterations for Reinforcement Learning Problems in Continuous Time and Space |
| Authors | Jae Young Lee, Richard S. Sutton |
| Abstract | Policy iteration (PI) is a recursive process of policy evaluation and improvement to solve an optimal decision-making, e.g., reinforcement learning (RL) or optimal control problem and has served as the fundamental to develop RL methods. Motivated by integral PI (IPI) schemes in optimal control and RL methods in continuous time and space (CTS), this paper proposes on-policy IPI to solve the general RL problem in CTS, with its environment modeled by an ordinary differential equation (ODE). In such continuous domain, we also propose four off-policy IPI methods—two are the ideal PI forms that use advantage and Q-functions, respectively, and the other two are natural extensions of the existing off-policy IPI schemes to our general RL framework. Compared to the IPI methods in optimal control, the proposed IPI schemes can be applied to more general situations and do not require an initial stabilizing policy to run; they are also strongly relevant to the RL algorithms in CTS such as advantage updating, Q-learning, and value-gradient based (VGB) greedy policy improvement. Our on-policy IPI is basically model-based but can be made partially model-free; each off-policy method is also either partially or completely model-free. The mathematical properties of the IPI methods—admissibility, monotone improvement, and convergence towards the optimal solution—are all rigorously proven, together with the equivalence of on- and off-policy IPI. Finally, the IPI methods are simulated with an inverted-pendulum model to support the theory and verify the performance. |
| Tasks | Decision Making, Q-Learning |
| Published | 2017-05-09 |
| URL | http://arxiv.org/abs/1705.03520v1 |
| http://arxiv.org/pdf/1705.03520v1.pdf | |
| PWC | https://paperswithcode.com/paper/integral-policy-iterations-for-reinforcement |
| Repo | |
| Framework | |
Online Learning for Structured Loss Spaces
| Title | Online Learning for Structured Loss Spaces |
| Authors | Siddharth Barman, Aditya Gopalan, Aadirupa Saha |
| Abstract | We consider prediction with expert advice when the loss vectors are assumed to lie in a set described by the sum of atomic norm balls. We derive a regret bound for a general version of the online mirror descent (OMD) algorithm that uses a combination of regularizers, each adapted to the constituent atomic norms. The general result recovers standard OMD regret bounds, and yields regret bounds for new structured settings where the loss vectors are (i) noisy versions of points from a low-rank subspace, (ii) sparse vectors corrupted with noise, and (iii) sparse perturbations of low-rank vectors. For the problem of online learning with structured losses, we also show lower bounds on regret in terms of rank and sparsity of the source set of the loss vectors, which implies lower bounds for the above additive loss settings as well. |
| Tasks | |
| Published | 2017-06-13 |
| URL | http://arxiv.org/abs/1706.04125v2 |
| http://arxiv.org/pdf/1706.04125v2.pdf | |
| PWC | https://paperswithcode.com/paper/online-learning-for-structured-loss-spaces |
| Repo | |
| Framework | |
Context-based Object Viewpoint Estimation: A 2D Relational Approach
| Title | Context-based Object Viewpoint Estimation: A 2D Relational Approach |
| Authors | Jose Oramas, Luc De Raedt, Tinne Tuytelaars |
| Abstract | The task of object viewpoint estimation has been a challenge since the early days of computer vision. To estimate the viewpoint (or pose) of an object, people have mostly looked at object intrinsic features, such as shape or appearance. Surprisingly, informative features provided by other, extrinsic elements in the scene, have so far mostly been ignored. At the same time, contextual cues have been proven to be of great benefit for related tasks such as object detection or action recognition. In this paper, we explore how information from other objects in the scene can be exploited for viewpoint estimation. In particular, we look at object configurations by following a relational neighbor-based approach for reasoning about object relations. We show that, starting from noisy object detections and viewpoint estimates, exploiting the estimated viewpoint and location of other objects in the scene can lead to improved object viewpoint predictions. Experiments on the KITTI dataset demonstrate that object configurations can indeed be used as a complementary cue to appearance-based viewpoint estimation. Our analysis reveals that the proposed context-based method can improve object viewpoint estimation by reducing specific types of viewpoint estimation errors commonly made by methods that only consider local information. Moreover, considering contextual information produces superior performance in scenes where a high number of object instances occur. Finally, our results suggest that, following a cautious relational neighbor formulation brings improvements over its aggressive counterpart for the task of object viewpoint estimation. |
| Tasks | Object Detection, Temporal Action Localization, Viewpoint Estimation |
| Published | 2017-04-21 |
| URL | http://arxiv.org/abs/1704.06610v1 |
| http://arxiv.org/pdf/1704.06610v1.pdf | |
| PWC | https://paperswithcode.com/paper/context-based-object-viewpoint-estimation-a |
| Repo | |
| Framework | |
DeepMasterPrints: Generating MasterPrints for Dictionary Attacks via Latent Variable Evolution
| Title | DeepMasterPrints: Generating MasterPrints for Dictionary Attacks via Latent Variable Evolution |
| Authors | Philip Bontrager, Aditi Roy, Julian Togelius, Nasir Memon, Arun Ross |
| Abstract | Recent research has demonstrated the vulnerability of fingerprint recognition systems to dictionary attacks based on MasterPrints. MasterPrints are real or synthetic fingerprints that can fortuitously match with a large number of fingerprints thereby undermining the security afforded by fingerprint systems. Previous work by Roy et al. generated synthetic MasterPrints at the feature-level. In this work we generate complete image-level MasterPrints known as DeepMasterPrints, whose attack accuracy is found to be much superior than that of previous methods. The proposed method, referred to as Latent Variable Evolution, is based on training a Generative Adversarial Network on a set of real fingerprint images. Stochastic search in the form of the Covariance Matrix Adaptation Evolution Strategy is then used to search for latent input variables to the generator network that can maximize the number of impostor matches as assessed by a fingerprint recognizer. Experiments convey the efficacy of the proposed method in generating DeepMasterPrints. The underlying method is likely to have broad applications in fingerprint security as well as fingerprint synthesis. |
| Tasks | |
| Published | 2017-05-21 |
| URL | http://arxiv.org/abs/1705.07386v4 |
| http://arxiv.org/pdf/1705.07386v4.pdf | |
| PWC | https://paperswithcode.com/paper/deepmasterprints-generating-masterprints-for |
| Repo | |
| Framework | |
Click Here: Human-Localized Keypoints as Guidance for Viewpoint Estimation
| Title | Click Here: Human-Localized Keypoints as Guidance for Viewpoint Estimation |
| Authors | Ryan Szeto, Jason J. Corso |
| Abstract | We motivate and address a human-in-the-loop variant of the monocular viewpoint estimation task in which the location and class of one semantic object keypoint is available at test time. In order to leverage the keypoint information, we devise a Convolutional Neural Network called Click-Here CNN (CH-CNN) that integrates the keypoint information with activations from the layers that process the image. It transforms the keypoint information into a 2D map that can be used to weigh features from certain parts of the image more heavily. The weighted sum of these spatial features is combined with global image features to provide relevant information to the prediction layers. To train our network, we collect a novel dataset of 3D keypoint annotations on thousands of CAD models, and synthetically render millions of images with 2D keypoint information. On test instances from PASCAL 3D+, our model achieves a mean class accuracy of 90.7%, whereas the state-of-the-art baseline only obtains 85.7% mean class accuracy, justifying our argument for human-in-the-loop inference. |
| Tasks | Viewpoint Estimation |
| Published | 2017-03-29 |
| URL | http://arxiv.org/abs/1703.09859v2 |
| http://arxiv.org/pdf/1703.09859v2.pdf | |
| PWC | https://paperswithcode.com/paper/click-here-human-localized-keypoints-as |
| Repo | |
| Framework | |
