Paper Group ANR 225
End-to-end training of object class detectors for mean average precision. Efficiently Creating 3D Training Data for Fine Hand Pose Estimation. evt_MNIST: A spike based version of traditional MNIST. Transfer Learning Based on AdaBoost for Feature Selection from Multiple ConvNet Layer Features. Enlightening Deep Neural Networks with Knowledge of Conf …
End-to-end training of object class detectors for mean average precision
| Title | End-to-end training of object class detectors for mean average precision |
| Authors | Paul Henderson, Vittorio Ferrari |
| Abstract | We present a method for training CNN-based object class detectors directly using mean average precision (mAP) as the training loss, in a truly end-to-end fashion that includes non-maximum suppression (NMS) at training time. This contrasts with the traditional approach of training a CNN for a window classification loss, then applying NMS only at test time, when mAP is used as the evaluation metric in place of classification accuracy. However, mAP following NMS forms a piecewise-constant structured loss over thousands of windows, with gradients that do not convey useful information for gradient descent. Hence, we define new, general gradient-like quantities for piecewise constant functions, which have wide applicability. We describe how to calculate these efficiently for mAP following NMS, enabling to train a detector based on Fast R-CNN directly for mAP. This model achieves equivalent performance to the standard Fast R-CNN on the PASCAL VOC 2007 and 2012 datasets, while being conceptually more appealing as the very same model and loss are used at both training and test time. |
| Tasks | |
| Published | 2016-07-12 |
| URL | http://arxiv.org/abs/1607.03476v2 |
| http://arxiv.org/pdf/1607.03476v2.pdf | |
| PWC | https://paperswithcode.com/paper/end-to-end-training-of-object-class-detectors |
| Repo | |
| Framework | |
Efficiently Creating 3D Training Data for Fine Hand Pose Estimation
| Title | Efficiently Creating 3D Training Data for Fine Hand Pose Estimation |
| Authors | Markus Oberweger, Gernot Riegler, Paul Wohlhart, Vincent Lepetit |
| Abstract | While many recent hand pose estimation methods critically rely on a training set of labelled frames, the creation of such a dataset is a challenging task that has been overlooked so far. As a result, existing datasets are limited to a few sequences and individuals, with limited accuracy, and this prevents these methods from delivering their full potential. We propose a semi-automated method for efficiently and accurately labeling each frame of a hand depth video with the corresponding 3D locations of the joints: The user is asked to provide only an estimate of the 2D reprojections of the visible joints in some reference frames, which are automatically selected to minimize the labeling work by efficiently optimizing a sub-modular loss function. We then exploit spatial, temporal, and appearance constraints to retrieve the full 3D poses of the hand over the complete sequence. We show that this data can be used to train a recent state-of-the-art hand pose estimation method, leading to increased accuracy. The code and dataset can be found on our website https://cvarlab.icg.tugraz.at/projects/hand_detection/ |
| Tasks | Hand Pose Estimation, Pose Estimation |
| Published | 2016-05-11 |
| URL | http://arxiv.org/abs/1605.03389v2 |
| http://arxiv.org/pdf/1605.03389v2.pdf | |
| PWC | https://paperswithcode.com/paper/efficiently-creating-3d-training-data-for |
| Repo | |
| Framework | |
evt_MNIST: A spike based version of traditional MNIST
| Title | evt_MNIST: A spike based version of traditional MNIST |
| Authors | Mazdak Fatahi, Mahmood Ahmadi, Mahyar Shahsavari, Arash Ahmadi, Philippe Devienne |
| Abstract | Benchmarks and datasets have important role in evaluation of machine learning algorithms and neural network implementations. Traditional dataset for images such as MNIST is applied to evaluate efficiency of different training algorithms in neural networks. This demand is different in Spiking Neural Networks (SNN) as they require spiking inputs. It is widely believed, in the biological cortex the timing of spikes is irregular. Poisson distributions provide adequate descriptions of the irregularity in generating appropriate spikes. Here, we introduce a spike-based version of MNSIT (handwritten digits dataset),using Poisson distribution and show the Poissonian property of the generated streams. We introduce a new version of evt_MNIST which can be used for neural network evaluation. |
| Tasks | |
| Published | 2016-04-22 |
| URL | http://arxiv.org/abs/1604.06751v1 |
| http://arxiv.org/pdf/1604.06751v1.pdf | |
| PWC | https://paperswithcode.com/paper/evt_mnist-a-spike-based-version-of |
| Repo | |
| Framework | |
Transfer Learning Based on AdaBoost for Feature Selection from Multiple ConvNet Layer Features
| Title | Transfer Learning Based on AdaBoost for Feature Selection from Multiple ConvNet Layer Features |
| Authors | Jumabek Alikhanov, Myeong Hyeon Ga, Seunghyun Ko, Geun-Sik Jo |
| Abstract | Convolutional Networks (ConvNets) are powerful models that learn hierarchies of visual features, which could also be used to obtain image representations for transfer learning. The basic pipeline for transfer learning is to first train a ConvNet on a large dataset (source task) and then use feed-forward units activation of the trained ConvNet as image representation for smaller datasets (target task). Our key contribution is to demonstrate superior performance of multiple ConvNet layer features over single ConvNet layer features. Combining multiple ConvNet layer features will result in more complex feature space with some features being repetitive. This requires some form of feature selection. We use AdaBoost with single stumps to implicitly select only distinct features that are useful towards classification from concatenated ConvNet features. Experimental results show that using multiple ConvNet layer activation features instead of single ConvNet layer features consistently will produce superior performance. Improvements becomes significant as we increase the distance between source task and the target task. |
| Tasks | Feature Selection, Transfer Learning |
| Published | 2016-02-01 |
| URL | http://arxiv.org/abs/1602.00417v2 |
| http://arxiv.org/pdf/1602.00417v2.pdf | |
| PWC | https://paperswithcode.com/paper/transfer-learning-based-on-adaboost-for |
| Repo | |
| Framework | |
Enlightening Deep Neural Networks with Knowledge of Confounding Factors
| Title | Enlightening Deep Neural Networks with Knowledge of Confounding Factors |
| Authors | Yu Zhong, Gil Ettinger |
| Abstract | Deep learning techniques have demonstrated significant capacity in modeling some of the most challenging real world problems of high complexity. Despite the popularity of deep models, we still strive to better understand the underlying mechanism that drives their success. Motivated by observations that neurons in trained deep nets predict attributes indirectly related to the training tasks, we recognize that a deep network learns representations more general than the task at hand to disentangle impacts of multiple confounding factors governing the data, in order to isolate the effects of the concerning factors and optimize a given objective. Consequently, we propose a general framework to augment training of deep models with information on auxiliary explanatory data variables, in an effort to boost this disentanglement and train deep networks that comprehend the data interactions and distributions more accurately, and thus improve their generalizability. We incorporate information on prominent auxiliary explanatory factors of the data population into existing architectures as secondary objective/loss blocks that take inputs from hidden layers during training. Once trained, these secondary circuits can be removed to leave a model with the same architecture as the original, but more generalizable and discerning thanks to its comprehension of data interactions. Since pose is one of the most dominant confounding factors for object recognition, we apply this principle to instantiate a pose-aware deep convolutional neural network and demonstrate that auxiliary pose information indeed improves the classification accuracy in our experiments on SAR target classification tasks. |
| Tasks | Object Recognition |
| Published | 2016-07-08 |
| URL | http://arxiv.org/abs/1607.02397v1 |
| http://arxiv.org/pdf/1607.02397v1.pdf | |
| PWC | https://paperswithcode.com/paper/enlightening-deep-neural-networks-with |
| Repo | |
| Framework | |
Online Low-Rank Subspace Learning from Incomplete Data: A Bayesian View
| Title | Online Low-Rank Subspace Learning from Incomplete Data: A Bayesian View |
| Authors | Paris V. Giampouras, Athanasios A. Rontogiannis, Konstantinos E. Themelis, Konstantinos D. Koutroumbas |
| Abstract | Extracting the underlying low-dimensional space where high-dimensional signals often reside has long been at the center of numerous algorithms in the signal processing and machine learning literature during the past few decades. At the same time, working with incomplete (partly observed) large scale datasets has recently been commonplace for diverse reasons. This so called {\it big data era} we are currently living calls for devising online subspace learning algorithms that can suitably handle incomplete data. Their envisaged objective is to {\it recursively} estimate the unknown subspace by processing streaming data sequentially, thus reducing computational complexity, while obviating the need for storing the whole dataset in memory. In this paper, an online variational Bayes subspace learning algorithm from partial observations is presented. To account for the unawareness of the true rank of the subspace, commonly met in practice, low-rankness is explicitly imposed on the sought subspace data matrix by exploiting sparse Bayesian learning principles. Moreover, sparsity, {\it simultaneously} to low-rankness, is favored on the subspace matrix by the sophisticated hierarchical Bayesian scheme that is adopted. In doing so, the proposed algorithm becomes adept in dealing with applications whereby the underlying subspace may be also sparse, as, e.g., in sparse dictionary learning problems. As shown, the new subspace tracking scheme outperforms its state-of-the-art counterparts in terms of estimation accuracy, in a variety of experiments conducted on simulated and real data. |
| Tasks | Dictionary Learning |
| Published | 2016-02-11 |
| URL | http://arxiv.org/abs/1602.03670v2 |
| http://arxiv.org/pdf/1602.03670v2.pdf | |
| PWC | https://paperswithcode.com/paper/online-low-rank-subspace-learning-from |
| Repo | |
| Framework | |
Efficient Attack Graph Analysis through Approximate Inference
| Title | Efficient Attack Graph Analysis through Approximate Inference |
| Authors | Luis Muñoz-González, Daniele Sgandurra, Andrea Paudice, Emil C. Lupu |
| Abstract | Attack graphs provide compact representations of the attack paths that an attacker can follow to compromise network resources by analysing network vulnerabilities and topology. These representations are a powerful tool for security risk assessment. Bayesian inference on attack graphs enables the estimation of the risk of compromise to the system’s components given their vulnerabilities and interconnections, and accounts for multi-step attacks spreading through the system. Whilst static analysis considers the risk posture at rest, dynamic analysis also accounts for evidence of compromise, e.g. from SIEM software or forensic investigation. However, in this context, exact Bayesian inference techniques do not scale well. In this paper we show how Loopy Belief Propagation - an approximate inference technique - can be applied to attack graphs, and that it scales linearly in the number of nodes for both static and dynamic analysis, making such analyses viable for larger networks. We experiment with different topologies and network clustering on synthetic Bayesian attack graphs with thousands of nodes to show that the algorithm’s accuracy is acceptable and converge to a stable solution. We compare sequential and parallel versions of Loopy Belief Propagation with exact inference techniques for both static and dynamic analysis, showing the advantages of approximate inference techniques to scale to larger attack graphs. |
| Tasks | Bayesian Inference |
| Published | 2016-06-22 |
| URL | http://arxiv.org/abs/1606.07025v1 |
| http://arxiv.org/pdf/1606.07025v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-attack-graph-analysis-through |
| Repo | |
| Framework | |
Measuring Sample Quality with Diffusions
| Title | Measuring Sample Quality with Diffusions |
| Authors | Jackson Gorham, Andrew B. Duncan, Sebastian J. Vollmer, Lester Mackey |
| Abstract | Stein’s method for measuring convergence to a continuous target distribution relies on an operator characterizing the target and Stein factor bounds on the solutions of an associated differential equation. While such operators and bounds are readily available for a diversity of univariate targets, few multivariate targets have been analyzed. We introduce a new class of characterizing operators based on Ito diffusions and develop explicit multivariate Stein factor bounds for any target with a fast-coupling Ito diffusion. As example applications, we develop computable and convergence-determining diffusion Stein discrepancies for log-concave, heavy-tailed, and multimodal targets and use these quality measures to select the hyperparameters of biased Markov chain Monte Carlo (MCMC) samplers, compare random and deterministic quadrature rules, and quantify bias-variance tradeoffs in approximate MCMC. Our results establish a near-linear relationship between diffusion Stein discrepancies and Wasserstein distances, improving upon past work even for strongly log-concave targets. The exposed relationship between Stein factors and Markov process coupling may be of independent interest. |
| Tasks | |
| Published | 2016-11-21 |
| URL | http://arxiv.org/abs/1611.06972v6 |
| http://arxiv.org/pdf/1611.06972v6.pdf | |
| PWC | https://paperswithcode.com/paper/measuring-sample-quality-with-diffusions |
| Repo | |
| Framework | |
Unsupervised Learning of Word-Sequence Representations from Scratch via Convolutional Tensor Decomposition
| Title | Unsupervised Learning of Word-Sequence Representations from Scratch via Convolutional Tensor Decomposition |
| Authors | Furong Huang, Animashree Anandkumar |
| Abstract | Unsupervised text embeddings extraction is crucial for text understanding in machine learning. Word2Vec and its variants have received substantial success in mapping words with similar syntactic or semantic meaning to vectors close to each other. However, extracting context-aware word-sequence embedding remains a challenging task. Training over large corpus is difficult as labels are difficult to get. More importantly, it is challenging for pre-trained models to obtain word-sequence embeddings that are universally good for all downstream tasks or for any new datasets. We propose a two-phased ConvDic+DeconvDec framework to solve the problem by combining a word-sequence dictionary learning model with a word-sequence embedding decode model. We propose a convolutional tensor decomposition mechanism to learn good word-sequence phrase dictionary in the learning phase. It is proved to be more accurate and much more efficient than the popular alternating minimization method. In the decode phase, we introduce a deconvolution framework that is immune to the problem of varying sentence lengths. The word-sequence embeddings we extracted using ConvDic+DeconvDec are universally good for a few downstream tasks we test on. The framework requires neither pre-training nor prior/outside information. |
| Tasks | Dictionary Learning |
| Published | 2016-06-10 |
| URL | http://arxiv.org/abs/1606.03153v3 |
| http://arxiv.org/pdf/1606.03153v3.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-learning-of-word-sequence |
| Repo | |
| Framework | |
Indirect Gaussian Graph Learning beyond Gaussianity
| Title | Indirect Gaussian Graph Learning beyond Gaussianity |
| Authors | Yiyuan She, Shao Tang, Qiaoya Zhang |
| Abstract | This paper studies how to capture dependency graph structures from real data which may not be Gaussian. Starting from marginal loss functions not necessarily derived from probability distributions, we utilize an additive over-parametrization with shrinkage to incorporate variable dependencies into the criterion. An iterative Gaussian graph learning algorithm is proposed with ease in implementation. Statistical analysis shows that the estimators achieve satisfactory accuracy with the error measured in terms of a proper Bregman divergence. Real-life examples in different settings are given to demonstrate the efficacy of the proposed methodology. |
| Tasks | |
| Published | 2016-10-08 |
| URL | https://arxiv.org/abs/1610.02590v4 |
| https://arxiv.org/pdf/1610.02590v4.pdf | |
| PWC | https://paperswithcode.com/paper/indirect-gaussian-graph-learning-beyond |
| Repo | |
| Framework | |
On higher order computations and synaptic meta-plasticity in the human brain: IT point of view (June, 2016)
| Title | On higher order computations and synaptic meta-plasticity in the human brain: IT point of view (June, 2016) |
| Authors | Stanislaw Ambroszkiewicz |
| Abstract | Glia modify neuronal connectivity by creating structural changes in the neuronal connectome. Glia also influence the functional connectome by modifying the flow of information through neural networks (Fields et al. 2015). There are strong experimental evidences that glia are responsible for synaptic meta-plasticity. Synaptic plasticity is the modification of the strength of connections between neurons. Meta-plasticity, i.e. plasticity of synaptic plasticity, may be viewed as mechanisms for dynamic reconfiguration of neuron circuits. First order computations in the brain are done by static neuron circuits, whereas higher order computations are done by dynamic reconfigurations of the links (synapses) between the neuron circuits. Static neuron circuits correspond to first order computable functions. Synapse creation correspond to the mathematical notion of function composition. Functionals are higher order functions that take functions as their arguments. The construction of functionals is based on dynamic reconfigurations of the function composition. Perhaps the functionals correspond to the meta-plasticity in the human brain. |
| Tasks | |
| Published | 2016-03-07 |
| URL | http://arxiv.org/abs/1603.02238v3 |
| http://arxiv.org/pdf/1603.02238v3.pdf | |
| PWC | https://paperswithcode.com/paper/on-higher-order-computations-and-synaptic |
| Repo | |
| Framework | |
One-Class SVM with Privileged Information and its Application to Malware Detection
| Title | One-Class SVM with Privileged Information and its Application to Malware Detection |
| Authors | Evgeny Burnaev, Dmitry Smolyakov |
| Abstract | A number of important applied problems in engineering, finance and medicine can be formulated as a problem of anomaly detection. A classical approach to the problem is to describe a normal state using a one-class support vector machine. Then to detect anomalies we quantify a distance from a new observation to the constructed description of the normal class. In this paper we present a new approach to the one-class classification. We formulate a new problem statement and a corresponding algorithm that allow taking into account a privileged information during the training phase. We evaluate performance of the proposed approach using a synthetic dataset, as well as the publicly available Microsoft Malware Classification Challenge dataset. |
| Tasks | Anomaly Detection, Malware Classification, Malware Detection |
| Published | 2016-09-26 |
| URL | http://arxiv.org/abs/1609.08039v2 |
| http://arxiv.org/pdf/1609.08039v2.pdf | |
| PWC | https://paperswithcode.com/paper/one-class-svm-with-privileged-information-and |
| Repo | |
| Framework | |
Mahalanobis Distance Metric Learning Algorithm for Instance-based Data Stream Classification
| Title | Mahalanobis Distance Metric Learning Algorithm for Instance-based Data Stream Classification |
| Authors | Jorge Luis Rivero Perez, Bernardete Ribeiro, Carlos Morell Perez |
| Abstract | With the massive data challenges nowadays and the rapid growing of technology, stream mining has recently received considerable attention. To address the large number of scenarios in which this phenomenon manifests itself suitable tools are required in various research fields. Instance-based data stream algorithms generally employ the Euclidean distance for the classification task underlying this problem. A novel way to look into this issue is to take advantage of a more flexible metric due to the increased requirements imposed by the data stream scenario. In this paper we present a new algorithm that learns a Mahalanobis metric using similarity and dissimilarity constraints in an online manner. This approach hybridizes a Mahalanobis distance metric learning algorithm and a k-NN data stream classification algorithm with concept drift detection. First, some basic aspects of Mahalanobis distance metric learning are described taking into account key properties as well as online distance metric learning algorithms. Second, we implement specific evaluation methodologies and comparative metrics such as Q statistic for data stream classification algorithms. Finally, our algorithm is evaluated on different datasets by comparing its results with one of the best instance-based data stream classification algorithm of the state of the art. The results demonstrate that our proposal is better |
| Tasks | Metric Learning |
| Published | 2016-04-17 |
| URL | http://arxiv.org/abs/1604.04879v1 |
| http://arxiv.org/pdf/1604.04879v1.pdf | |
| PWC | https://paperswithcode.com/paper/mahalanobis-distance-metric-learning |
| Repo | |
| Framework | |
Image-Text Multi-Modal Representation Learning by Adversarial Backpropagation
| Title | Image-Text Multi-Modal Representation Learning by Adversarial Backpropagation |
| Authors | Gwangbeen Park, Woobin Im |
| Abstract | We present novel method for image-text multi-modal representation learning. In our knowledge, this work is the first approach of applying adversarial learning concept to multi-modal learning and not exploiting image-text pair information to learn multi-modal feature. We only use category information in contrast with most previous methods using image-text pair information for multi-modal embedding. In this paper, we show that multi-modal feature can be achieved without image-text pair information and our method makes more similar distribution with image and text in multi-modal feature space than other methods which use image-text pair information. And we show our multi-modal feature has universal semantic information, even though it was trained for category prediction. Our model is end-to-end backpropagation, intuitive and easily extended to other multi-modal learning work. |
| Tasks | Representation Learning |
| Published | 2016-12-26 |
| URL | http://arxiv.org/abs/1612.08354v1 |
| http://arxiv.org/pdf/1612.08354v1.pdf | |
| PWC | https://paperswithcode.com/paper/image-text-multi-modal-representation |
| Repo | |
| Framework | |
A new class of metrics for learning on real-valued and structured data
| Title | A new class of metrics for learning on real-valued and structured data |
| Authors | Ruiyu Yang, Yuxiang Jiang, Scott Mathews, Elizabeth A. Housworth, Matthew W. Hahn, Predrag Radivojac |
| Abstract | We propose a new class of metrics on sets, vectors, and functions that can be used in various stages of data mining, including exploratory data analysis, learning, and result interpretation. These new distance functions unify and generalize some of the popular metrics, such as the Jaccard and bag distances on sets, Manhattan distance on vector spaces, and Marczewski-Steinhaus distance on integrable functions. We prove that the new metrics are complete and show useful relationships with $f$-divergences for probability distributions. To further extend our approach to structured objects such as concept hierarchies and ontologies, we introduce information-theoretic metrics on directed acyclic graphs drawn according to a fixed probability distribution. We conduct empirical investigation to demonstrate intuitive interpretation of the new metrics and their effectiveness on real-valued, high-dimensional, and structured data. Extensive comparative evaluation demonstrates that the new metrics outperformed multiple similarity and dissimilarity functions traditionally used in data mining, including the Minkowski family, the fractional $L^p$ family, two $f$-divergences, cosine distance, and two correlation coefficients. Finally, we argue that the new class of metrics is particularly appropriate for rapid processing of high-dimensional and structured data in distance-based learning. |
| Tasks | |
| Published | 2016-03-22 |
| URL | http://arxiv.org/abs/1603.06846v3 |
| http://arxiv.org/pdf/1603.06846v3.pdf | |
| PWC | https://paperswithcode.com/paper/a-new-class-of-metrics-for-learning-on-real |
| Repo | |
| Framework | |
