Paper Group ANR 1603
The statistical Minkowski distances: Closed-form formula for Gaussian Mixture Models. Single Image based Head Pose Estimation with Spherical Parameterization and 3D Morphing. Towards Real-Time Head Pose Estimation: Exploring Parameter-Reduced Residual Networks on In-the-wild Datasets. Deep Features for Tissue-Fold Detection in Histopathology Images …
The statistical Minkowski distances: Closed-form formula for Gaussian Mixture Models
| Title | The statistical Minkowski distances: Closed-form formula for Gaussian Mixture Models |
| Authors | Frank Nielsen |
| Abstract | The traditional Minkowski distances are induced by the corresponding Minkowski norms in real-valued vector spaces. In this work, we propose novel statistical symmetric distances based on the Minkowski’s inequality for probability densities belonging to Lebesgue spaces. These statistical Minkowski distances admit closed-form formula for Gaussian mixture models when parameterized by integer exponents. This result extends to arbitrary mixtures of exponential families with natural parameter spaces being cones: This includes the binomial, the multinomial, the zero-centered Laplacian, the Gaussian and the Wishart mixtures, among others. We also derive a Minkowski’s diversity index of a normalized weighted set of probability distributions from Minkowski’s inequality. |
| Tasks | |
| Published | 2019-01-09 |
| URL | http://arxiv.org/abs/1901.03732v2 |
| http://arxiv.org/pdf/1901.03732v2.pdf | |
| PWC | https://paperswithcode.com/paper/the-statistical-minkowski-distances-closed |
| Repo | |
| Framework | |
Single Image based Head Pose Estimation with Spherical Parameterization and 3D Morphing
| Title | Single Image based Head Pose Estimation with Spherical Parameterization and 3D Morphing |
| Authors | Hui Yuan, Mengyu Li, Junhui Hou, Jimin Xiao |
| Abstract | Head pose estimation plays a vital role in various applications, e.g., driverassistance systems, human-computer interaction, virtual reality technology, and so on. We propose a novel geometry based algorithm for accurately estimating the head pose from a single 2D face image at a very low computational cost. Specifically, the rectangular coordinates of only four non-coplanar feature points from a predefined 3D facial model as well as the corresponding ones automatically/ manually extracted from a 2D face image are first normalized to exclude the effect of external factors (i.e., scale factor and translation parameters). Then, the four normalized 3D feature points are represented in spherical coordinates with reference to the uniquely determined sphere by themselves. Due to the spherical parameterization, the coordinates of feature points can then be morphed along all the three directions in the rectangular coordinates effectively. Finally, the rotation matrix indicating the head pose is obtained by minimizing the Euclidean distance between the normalized 2D feature points and the 2D re-projections of morphed 3D feature points. Comprehensive experimental results over two popular databases, i.e., Pointing’04 and Biwi Kinect, demonstrate that the proposed algorithm can estimate head poses with higher accuracy and lower run time than state-of-the-art geometry based methods. Even compared with start-of-the-art learning based methods or geometry based methods with additional depth information, our algorithm still produces comparable performance. |
| Tasks | Head Pose Estimation, Pose Estimation |
| Published | 2019-07-22 |
| URL | https://arxiv.org/abs/1907.09217v3 |
| https://arxiv.org/pdf/1907.09217v3.pdf | |
| PWC | https://paperswithcode.com/paper/a-single-image-based-head-pose-estimation |
| Repo | |
| Framework | |
Towards Real-Time Head Pose Estimation: Exploring Parameter-Reduced Residual Networks on In-the-wild Datasets
| Title | Towards Real-Time Head Pose Estimation: Exploring Parameter-Reduced Residual Networks on In-the-wild Datasets |
| Authors | Ines Rieger, Thomas Hauenstein, Sebastian Hettenkofer, Jens-Uwe Garbas |
| Abstract | Head poses are a key component of human bodily communication and thus a decisive element of human-computer interaction. Real-time head pose estimation is crucial in the context of human-robot interaction or driver assistance systems. The most promising approaches for head pose estimation are based on Convolutional Neural Networks (CNNs). However, CNN models are often too complex to achieve real-time performance. To face this challenge, we explore a popular subgroup of CNNs, the Residual Networks (ResNets) and modify them in order to reduce their number of parameters. The ResNets are modifed for different image sizes including low-resolution images and combined with a varying number of layers. They are trained on in-the-wild datasets to ensure real-world applicability. As a result, we demonstrate that the performance of the ResNets can be maintained while reducing the number of parameters. The modified ResNets achieve state-of-the-art accuracy and provide fast inference for real-time applicability. |
| Tasks | Head Pose Estimation, Pose Estimation |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.05203v2 |
| https://arxiv.org/pdf/1906.05203v2.pdf | |
| PWC | https://paperswithcode.com/paper/towards-real-time-head-pose-estimation |
| Repo | |
| Framework | |
Deep Features for Tissue-Fold Detection in Histopathology Images
| Title | Deep Features for Tissue-Fold Detection in Histopathology Images |
| Authors | Morteza Babaie, H. R. Tizhoosh |
| Abstract | Whole slide imaging (WSI) refers to the digitization of a tissue specimen which enables pathologists to explore high-resolution images on a monitor rather than through a microscope. The formation of tissue folds occur during tissue processing. Their presence may not only cause out-of-focus digitization but can also negatively affect the diagnosis in some cases. In this paper, we have compared five pre-trained convolutional neural networks (CNNs) of different depths as feature extractors to characterize tissue folds. We have also explored common classifiers to discriminate folded tissue against the normal tissue in hematoxylin and eosin (H&E) stained biopsy samples. In our experiments, we manually select the folded area in roughly 2.5mm $\times$ 2.5mm patches at $20$x magnification level as the training data. The ``DenseNet’’ with 201 layers alongside an SVM classifier outperformed all other configurations. Based on the leave-one-out validation strategy, we achieved $96.3%$ accuracy, whereas with augmentation the accuracy increased to $97.2%$. We have tested the generalization of our method with five unseen WSIs from the NIH (National Cancer Institute) dataset. The accuracy for patch-wise detection was $81%$. One folded patch within an image suffices to flag the entire specimen for visual inspection. | |
| Tasks | |
| Published | 2019-03-17 |
| URL | http://arxiv.org/abs/1903.07011v1 |
| http://arxiv.org/pdf/1903.07011v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-features-for-tissue-fold-detection-in |
| Repo | |
| Framework | |
How to “DODGE” Complex Software Analytics?
| Title | How to “DODGE” Complex Software Analytics? |
| Authors | Amritanshu Agrawal, Wei Fu, Di Chen, Xipeng Shen, Tim Menzies |
| Abstract | Machine learning techniques applied to software engineering tasks can be improved by hyperparameter optimization, i.e., automatic tools that find good settings for a learner’s control parameters. We show that such hyperparameter optimization can be unnecessarily slow, particularly when the optimizers waste time exploring “redundant tunings”', i.e., pairs of tunings which lead to indistinguishable results. By ignoring redundant tunings, DODGE, a tuning tool, runs orders of magnitude faster, while also generating learners with more accurate predictions than seen in prior state-of-the-art approaches. |
| Tasks | Hyperparameter Optimization |
| Published | 2019-02-05 |
| URL | https://arxiv.org/abs/1902.01838v2 |
| https://arxiv.org/pdf/1902.01838v2.pdf | |
| PWC | https://paperswithcode.com/paper/how-to-dodge-complex-software-analytics |
| Repo | |
| Framework | |
BioSNet: A Fast-Learning and High-Robustness Unsupervised Biomimetic Spiking Neural Network
| Title | BioSNet: A Fast-Learning and High-Robustness Unsupervised Biomimetic Spiking Neural Network |
| Authors | Mingyuan Meng, Xingyu Yang, Shanlin Xiao, Zhiyi Yu |
| Abstract | Spiking Neural Network (SNN), as a brain-inspired machine learning algorithm, is closer to the computing mechanism of human brain and more suitable to reveal the essence of intelligence compared with Artificial Neural Networks (ANN), attracting more and more attention in recent years. In addition, the information processed by SNN is in the form of discrete spikes, which makes SNN have low power consumption characteristics. In this paper, we propose an efficient and strong unsupervised SNN named BioSNet with high biological plausibility to handle image classification tasks. In BioSNet, we propose a new biomimetic spiking neuron model named MRON inspired by ‘recognition memory’ in the human brain, design an efficient and robust network architecture corresponding to biological characteristics of the human brain as well, and extend the traditional voting mechanism to the Vote-for-All (VFA) decoding layer so as to reduce information loss during decoding. Simulation results show that BioSNet not only achieves state-of-the-art unsupervised classification accuracy on MNIST/EMNIST data sets, but also exhibits superior learning efficiency and high robustness. Specifically, the BioSNet trained with only dozens of samples per class can achieve a favorable classification accuracy over 80% and randomly deleting even 95% of synapses or neurons in the BioSNet only leads to slight performance degradation. |
| Tasks | Image Classification |
| Published | 2019-12-02 |
| URL | https://arxiv.org/abs/2001.01680v2 |
| https://arxiv.org/pdf/2001.01680v2.pdf | |
| PWC | https://paperswithcode.com/paper/biosnet-a-fast-learning-and-high-robustness |
| Repo | |
| Framework | |
Fast Neural Network Approach for Direct Covariant Forces Prediction in Complex Multi-Element Extended Systems
| Title | Fast Neural Network Approach for Direct Covariant Forces Prediction in Complex Multi-Element Extended Systems |
| Authors | Jonathan P. Mailoa, Mordechai Kornbluth, Simon L. Batzner, Georgy Samsonidze, Stephen T. Lam, Chris Ablitt, Nicola Molinari, Boris Kozinsky |
| Abstract | Neural network force field (NNFF) is a method for performing regression on atomic structure-force relationships, bypassing expensive quantum mechanics calculation which prevents the execution of long ab-initio quality molecular dynamics simulations. However, most NNFF methods for complex multi-element atomic systems indirectly predict atomic force vectors by exploiting just atomic structure rotation-invariant features and the network-feature spatial derivatives which are computationally expensive. We develop a staggered NNFF architecture exploiting both rotation-invariant and covariant features separately to directly predict atomic force vectors without using spatial derivatives, thereby reducing expensive structural feature calculation by ~180-480x. This acceleration enables us to develop NNFF which directly predicts atomic forces in complex ternary and quaternary-element extended systems comprised of long polymer chains, amorphous oxide, and surface chemical reactions. The staggered rotation-invariant-covariant architecture described here can also directly predict complex covariant vector outputs from local physical structures in domains beyond computational material science. |
| Tasks | |
| Published | 2019-05-07 |
| URL | https://arxiv.org/abs/1905.02791v1 |
| https://arxiv.org/pdf/1905.02791v1.pdf | |
| PWC | https://paperswithcode.com/paper/fast-neural-network-approach-for-direct |
| Repo | |
| Framework | |
Frequentist Regret Bounds for Randomized Least-Squares Value Iteration
| Title | Frequentist Regret Bounds for Randomized Least-Squares Value Iteration |
| Authors | Andrea Zanette, David Brandfonbrener, Emma Brunskill, Matteo Pirotta, Alessandro Lazaric |
| Abstract | We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning (RL). When the state space is large or continuous, traditional tabular approaches are unfeasible and some form of function approximation is mandatory. In this paper, we introduce an optimistically-initialized variant of the popular randomized least-squares value iteration (RLSVI), a model-free algorithm where exploration is induced by perturbing the least-squares approximation of the action-value function. Under the assumption that the Markov decision process has low-rank transition dynamics, we prove that the frequentist regret of RLSVI is upper-bounded by $\widetilde O(d^2 H^2 \sqrt{T})$ where $ d $ are the feature dimension, $ H $ is the horizon, and $ T $ is the total number of steps. To the best of our knowledge, this is the first frequentist regret analysis for randomized exploration with function approximation. |
| Tasks | |
| Published | 2019-11-01 |
| URL | https://arxiv.org/abs/1911.00567v4 |
| https://arxiv.org/pdf/1911.00567v4.pdf | |
| PWC | https://paperswithcode.com/paper/frequentist-regret-bounds-for-randomized |
| Repo | |
| Framework | |
On Uniform Equivalence of Epistemic Logic Programs
| Title | On Uniform Equivalence of Epistemic Logic Programs |
| Authors | Wolfgang Faber, Michael Morak, Stefan Woltran |
| Abstract | Epistemic Logic Programs (ELPs) extend Answer Set Programming (ASP) with epistemic negation and have received renewed interest in recent years. This led to the development of new research and efficient solving systems for ELPs. In practice, ELPs are often written in a modular way, where each module interacts with other modules by accepting sets of facts as input, and passing on sets of facts as output. An interesting question then presents itself: under which conditions can such a module be replaced by another one without changing the outcome, for any set of input facts? This problem is known as uniform equivalence, and has been studied extensively for ASP. For ELPs, however, such an investigation is, as of yet, missing. In this paper, we therefore propose a characterization of uniform equivalence that can be directly applied to the language of state-of-the-art ELP solvers. We also investigate the computational complexity of deciding uniform equivalence for two ELPs, and show that it is on the third level of the polynomial hierarchy. |
| Tasks | |
| Published | 2019-07-25 |
| URL | https://arxiv.org/abs/1907.10925v1 |
| https://arxiv.org/pdf/1907.10925v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-uniform-equivalence-of-epistemic-logic |
| Repo | |
| Framework | |
Consistent Risk Estimation in High-Dimensional Linear Regression
| Title | Consistent Risk Estimation in High-Dimensional Linear Regression |
| Authors | Ji Xu, Arian Maleki, Kamiar Rahnama Rad |
| Abstract | Risk estimation is at the core of many learning systems. The importance of this problem has motivated researchers to propose different schemes, such as cross validation, generalized cross validation, and Bootstrap. The theoretical properties of such estimates have been extensively studied in the low-dimensional settings, where the number of predictors $p$ is much smaller than the number of observations $n$. However, a unifying methodology accompanied with a rigorous theory is lacking in high-dimensional settings. This paper studies the problem of risk estimation under the high-dimensional asymptotic setting $n,p \rightarrow \infty$ and $n/p \rightarrow \delta$ ($\delta$ is a fixed number), and proves the consistency of three risk estimates that have been successful in numerical studies, i.e., leave-one-out cross validation (LOOCV), approximate leave-one-out (ALO), and approximate message passing (AMP)-based techniques. A corner stone of our analysis is a bound that we obtain on the discrepancy of the `residuals’ obtained from AMP and LOOCV. This connection not only enables us to obtain a more refined information on the estimates of AMP, ALO, and LOOCV, but also offers an upper bound on the convergence rate of each estimate. | |
| Tasks | |
| Published | 2019-02-05 |
| URL | http://arxiv.org/abs/1902.01753v2 |
| http://arxiv.org/pdf/1902.01753v2.pdf | |
| PWC | https://paperswithcode.com/paper/consistent-risk-estimation-in-high |
| Repo | |
| Framework | |
Trading-Off Static and Dynamic Regret in Online Least-Squares and Beyond
| Title | Trading-Off Static and Dynamic Regret in Online Least-Squares and Beyond |
| Authors | Jianjun Yuan, Andrew Lamperski |
| Abstract | Recursive least-squares algorithms often use forgetting factors as a heuristic to adapt to non-stationary data streams. The first contribution of this paper rigorously characterizes the effect of forgetting factors for a class of online Newton algorithms. For exp-concave and strongly convex objectives, the algorithms achieve the dynamic regret of $\max{O(\log T),O(\sqrt{TV})}$, where $V$ is a bound on the path length of the comparison sequence. In particular, we show how classic recursive least-squares with a forgetting factor achieves this dynamic regret bound. By varying $V$, we obtain a trade-off between static and dynamic regret. In order to obtain more computationally efficient algorithms, our second contribution is a novel gradient descent step size rule for strongly convex functions. Our gradient descent rule recovers the order optimal dynamic regret bounds described above. For smooth problems, we can also obtain static regret of $O(T^{1-\beta})$ and dynamic regret of $O(T^\beta V^*)$, where $\beta \in (0,1)$ and $V^*$ is the path length of the sequence of minimizers. By varying $\beta$, we obtain a trade-off between static and dynamic regret. |
| Tasks | |
| Published | 2019-09-06 |
| URL | https://arxiv.org/abs/1909.03118v2 |
| https://arxiv.org/pdf/1909.03118v2.pdf | |
| PWC | https://paperswithcode.com/paper/trading-off-static-and-dynamic-regret-in |
| Repo | |
| Framework | |
Recommendations on Designing Practical Interval Type-2 Fuzzy Systems
| Title | Recommendations on Designing Practical Interval Type-2 Fuzzy Systems |
| Authors | Dongrui Wu, Jerry Mendel |
| Abstract | Interval type-2 (IT2) fuzzy systems have become increasingly popular in the last 20 years. They have demonstrated superior performance in many applications. However, the operation of an IT2 fuzzy system is more complex than that of its type-1 counterpart. There are many questions to be answered in designing an IT2 fuzzy system: Should singleton or non-singleton fuzzifier be used? How many membership functions (MFs) should be used for each input? Should Gaussian or piecewise linear MFs be used? Should Mamdani or Takagi-Sugeno-Kang (TSK) inference be used? Should minimum or product $t$-norm be used? Should type-reduction be used or not? How to optimize the IT2 fuzzy system? These questions may look overwhelming and confusing to IT2 beginners. In this paper we recommend some representative starting choices for an IT2 fuzzy system design, which hopefully will make IT2 fuzzy systems more accessible to IT2 fuzzy system designers. |
| Tasks | |
| Published | 2019-07-03 |
| URL | https://arxiv.org/abs/1907.01697v1 |
| https://arxiv.org/pdf/1907.01697v1.pdf | |
| PWC | https://paperswithcode.com/paper/recommendations-on-designing-practical |
| Repo | |
| Framework | |
Incremental Improvement of a Question Answering System by Re-ranking Answer Candidates using Machine Learning
| Title | Incremental Improvement of a Question Answering System by Re-ranking Answer Candidates using Machine Learning |
| Authors | Michael Barz, Daniel Sonntag |
| Abstract | We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. We focus on improving deployed QA systems that do not allow re-training or re-training comes at a high cost. Our re-ranking approach learns a similarity function using n-gram based features using the query, the answer and the initial system confidence as input. Our contributions are: (1) we generate a QA training corpus starting from 877 answers from the customer care domain of T-Mobile Austria, (2) we implement a state-of-the-art QA pipeline using neural sentence embeddings that encode queries in the same space than the answer index, and (3) we evaluate the QA pipeline and our re-ranking approach using a separately provided test set. The test set can be considered to be available after deployment of the system, e.g., based on feedback of users. Our results show that the system performance, in terms of top-n accuracy and the mean reciprocal rank, benefits from re-ranking using gradient boosted regression trees. On average, the mean reciprocal rank improves by 9.15%. |
| Tasks | Answer Selection, Question Answering, Sentence Embeddings |
| Published | 2019-08-27 |
| URL | https://arxiv.org/abs/1908.10149v1 |
| https://arxiv.org/pdf/1908.10149v1.pdf | |
| PWC | https://paperswithcode.com/paper/incremental-improvement-of-a-question |
| Repo | |
| Framework | |
Active Learning for Black-Box Adversarial Attacks in EEG-Based Brain-Computer Interfaces
| Title | Active Learning for Black-Box Adversarial Attacks in EEG-Based Brain-Computer Interfaces |
| Authors | Xue Jiang, Xiao Zhang, Dongrui Wu |
| Abstract | Deep learning has made significant breakthroughs in many fields, including electroencephalogram (EEG) based brain-computer interfaces (BCIs). However, deep learning models are vulnerable to adversarial attacks, in which deliberately designed small perturbations are added to the benign input samples to fool the deep learning model and degrade its performance. This paper considers transferability-based black-box attacks, where the attacker trains a substitute model to approximate the target model, and then generates adversarial examples from the substitute model to attack the target model. Learning a good substitute model is critical to the success of these attacks, but it requires a large number of queries to the target model. We propose a novel framework which uses query synthesis based active learning to improve the query efficiency in training the substitute model. Experiments on three convolutional neural network (CNN) classifiers and three EEG datasets demonstrated that our method can improve the attack success rate with the same number of queries, or, in other words, our method requires fewer queries to achieve a desired attack performance. To our knowledge, this is the first work that integrates active learning and adversarial attacks for EEG-based BCIs. |
| Tasks | Active Learning, EEG |
| Published | 2019-11-07 |
| URL | https://arxiv.org/abs/1911.04338v1 |
| https://arxiv.org/pdf/1911.04338v1.pdf | |
| PWC | https://paperswithcode.com/paper/active-learning-for-black-box-adversarial |
| Repo | |
| Framework | |
Sequential Learning of Active Subspaces
| Title | Sequential Learning of Active Subspaces |
| Authors | Nathan Wycoff, Mickael Binois, Stefan M. Wild |
| Abstract | In recent years, active subspace methods (ASMs) have become a popular means of performing subspace sensitivity analysis on black-box functions. Naively applied, however, ASMs require gradient evaluations of the target function. In the event of noisy, expensive, or stochastic simulators, evaluating gradients via finite differencing may be infeasible. In such cases, often a surrogate model is employed, on which finite differencing is performed. When the surrogate model is a Gaussian process, we show that the ASM estimator is available in closed form, rendering the finite-difference approximation unnecessary. We use our closed-form solution to develop acquisition functions focused on sequential learning tailored to sensitivity analysis on top of ASMs. We also show that the traditional ASM estimator may be viewed as a method of moments estimator for a certain class of Gaussian processes. We demonstrate how uncertainty on Gaussian process hyperparameters may be propagated to uncertainty on the sensitivity analysis, allowing model-based confidence intervals on the active subspace. Our methodological developments are illustrated on several examples. |
| Tasks | Gaussian Processes |
| Published | 2019-07-26 |
| URL | https://arxiv.org/abs/1907.11572v1 |
| https://arxiv.org/pdf/1907.11572v1.pdf | |
| PWC | https://paperswithcode.com/paper/sequential-learning-of-active-subspaces |
| Repo | |
| Framework | |
