Paper Group ANR 5
Aleatoric and Epistemic Uncertainty in Machine Learning: A Tutorial Introduction. AlgoNet: $C^\infty$ Smooth Algorithmic Neural Networks. Exploring Temporal Differences in 3D Convolutional Neural Networks. Convolutional Neural Networks: A Binocular Vision Perspective. Flexible Clustering with a Sparse Mixture of Generalized Hyperbolic Distributions …
Aleatoric and Epistemic Uncertainty in Machine Learning: A Tutorial Introduction
| Title | Aleatoric and Epistemic Uncertainty in Machine Learning: A Tutorial Introduction |
| Authors | Eyke Hüllermeier, Willem Waegeman |
| Abstract | The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology. In line with the statistical tradition, uncertainty has long been perceived as almost synonymous with standard probability and probabilistic predictions. Yet, due to the steadily increasing relevance of machine learning for practical applications and related issues such as safety requirements, new problems and challenges have recently been identified by machine learning scholars, and these problems may call for new methodological developments. In particular, this includes the importance of distinguishing between (at least) two different types of uncertainty, often refereed to as aleatoric and epistemic. In this paper, we provide an introduction to the topic of uncertainty in machine learning as well as an overview of hitherto attempts at handling uncertainty in general and formalizing this distinction in particular. |
| Tasks | |
| Published | 2019-10-21 |
| URL | https://arxiv.org/abs/1910.09457v1 |
| https://arxiv.org/pdf/1910.09457v1.pdf | |
| PWC | https://paperswithcode.com/paper/aleatoric-and-epistemic-uncertainty-in |
| Repo | |
| Framework | |
AlgoNet: $C^\infty$ Smooth Algorithmic Neural Networks
| Title | AlgoNet: $C^\infty$ Smooth Algorithmic Neural Networks |
| Authors | Felix Petersen, Christian Borgelt, Oliver Deussen |
| Abstract | Artificial neural networks revolutionized many areas of computer science in recent years since they provide solutions to a number of previously unsolved problems. On the other hand, for many problems, classic algorithms exist, which typically exceed the accuracy and stability of neural networks. To combine these two concepts, we present a new kind of neural networks$-$algorithmic neural networks (AlgoNets). These networks integrate smooth versions of classic algorithms into the topology of neural networks. A forward AlgoNet includes algorithmic layers into existing architectures while a backward AlgoNet can solve inverse problems without or with only weak supervision. In addition, we present the $\texttt{algonet}$ package, a PyTorch based library that includes, inter alia, a smoothly evaluated programming language, a smooth 3D mesh renderer, and smooth sorting algorithms. |
| Tasks | |
| Published | 2019-05-16 |
| URL | https://arxiv.org/abs/1905.06886v2 |
| https://arxiv.org/pdf/1905.06886v2.pdf | |
| PWC | https://paperswithcode.com/paper/algonet-cinfty-smooth-algorithmic-neural |
| Repo | |
| Framework | |
Exploring Temporal Differences in 3D Convolutional Neural Networks
| Title | Exploring Temporal Differences in 3D Convolutional Neural Networks |
| Authors | Gagan Kanojia, Sudhakar Kumawat, Shanmuganathan Raman |
| Abstract | Traditional 3D convolutions are computationally expensive, memory intensive, and due to large number of parameters, they often tend to overfit. On the other hand, 2D CNNs are less computationally expensive and less memory intensive than 3D CNNs and have shown remarkable results in applications like image classification and object recognition. However, in previous works, it has been observed that they are inferior to 3D CNNs when applied on a spatio-temporal input. In this work, we propose a convolutional block which extracts the spatial information by performing a 2D convolution and extracts the temporal information by exploiting temporal differences, i.e., the change in the spatial information at different time instances, using simple operations of shift, subtract and add without utilizing any trainable parameters. The proposed convolutional block has same number of parameters as of a 2D convolution kernel of size nxn, i.e. n^2, and has n times lesser parameters than an nxnxn 3D convolution kernel. We show that the 3D CNNs perform better when the 3D convolution kernels are replaced by the proposed convolutional blocks. We evaluate the proposed convolutional block on UCF101 and ModelNet datasets. |
| Tasks | Image Classification, Object Recognition |
| Published | 2019-09-07 |
| URL | https://arxiv.org/abs/1909.03309v1 |
| https://arxiv.org/pdf/1909.03309v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-temporal-differences-in-3d |
| Repo | |
| Framework | |
Convolutional Neural Networks: A Binocular Vision Perspective
| Title | Convolutional Neural Networks: A Binocular Vision Perspective |
| Authors | Yigit Oktar, Diclehan Karakaya, Oguzhan Ulucan, Mehmet Turkan |
| Abstract | It is arguable that whether the single camera captured (monocular) image datasets are sufficient enough to train and test convolutional neural networks (CNNs) for imitating the biological neural network structures of the human brain. As human visual system works in binocular, the collaboration of the eyes with the two brain lobes needs more investigation for improvements in such CNN-based visual imagery analysis applications. It is indeed questionable that if respective visual fields of each eye and the associated brain lobes are responsible for different learning abilities of the same scene. There are such open questions in this field of research which need rigorous investigation in order to further understand the nature of the human visual system, hence improve the currently available deep learning applications. This position paper analyses a binocular CNNs architecture that is more analogous to the biological structure of the human visual system than the conventional deep learning techniques. While taking a structure called optic chiasma into account, this architecture consists of basically two parallel CNN structures associated with each visual field and the brain lobe, fully connected later possibly as in the primary visual cortex (V1). Experimental results demonstrate that binocular learning of two different visual fields leads to better classification rates on average, when compared to classical CNN architectures. |
| Tasks | |
| Published | 2019-12-21 |
| URL | https://arxiv.org/abs/1912.10201v1 |
| https://arxiv.org/pdf/1912.10201v1.pdf | |
| PWC | https://paperswithcode.com/paper/convolutional-neural-networks-a-binocular |
| Repo | |
| Framework | |
Flexible Clustering with a Sparse Mixture of Generalized Hyperbolic Distributions
| Title | Flexible Clustering with a Sparse Mixture of Generalized Hyperbolic Distributions |
| Authors | Michael P. B. Gallaugher, Yang Tang, Paul D. McNicholas |
| Abstract | Robust clustering of high-dimensional data is an important topic because, in many practical situations, real data sets are heavy-tailed and/or asymmetric. Moreover, traditional model-based clustering often fails for high dimensional data due to the number of free covariance parameters. A parametrization of the component scale matrices for the mixture of generalized hyperbolic distributions is proposed by including a penalty term in the likelihood constraining the parameters resulting in a flexible model for high dimensional data and a meaningful interpretation. An analytically feasible EM algorithm is developed by placing a gamma-Lasso penalty constraining the concentration matrix. The proposed methodology is investigated through simulation studies and two real data sets. |
| Tasks | |
| Published | 2019-03-12 |
| URL | http://arxiv.org/abs/1903.05054v1 |
| http://arxiv.org/pdf/1903.05054v1.pdf | |
| PWC | https://paperswithcode.com/paper/flexible-clustering-with-a-sparse-mixture-of |
| Repo | |
| Framework | |
Decoupling Features and Coordinates for Few-shot RGB Relocalization
| Title | Decoupling Features and Coordinates for Few-shot RGB Relocalization |
| Authors | Siyan Dong, Songyin Wu, Yixin Zhuang, Shanghang Zhang, Kai Xu, Baoquan Chen |
| Abstract | Cross-scene model adaption is a crucial feature for camera relocalization applied in real scenarios. It is preferable that a pre-learned model can be quickly deployed in a novel scene with as little training as possible. The existing state-of-the-art approaches, however, can hardly support few-shot scene adaption due to the entangling of image feature extraction and 3D coordinate regression, which requires a large-scale of training data. To address this issue, inspired by how humans relocalize, we approach camera relocalization with a decoupled solution where feature extraction, coordinate regression and pose estimation are performed separately. Our key insight is that robust and discriminative image features used for coordinate regression should be learned by removing the distracting factor of camera views, because coordinates in the world reference frame are obviously independent of local views. In particular, we employ a deep neural network to learn view-factorized pixel-wise features using several training scenes. Given a new scene, we train a view-dependent per-pixel 3D coordinate regressor while keeping the feature extractor fixed. Such a decoupled design allows us to adapt the entire model to novel scene and achieve accurate camera pose estimation with only few-shot training samples and two orders of magnitude less training time than the state-of-the-arts. |
| Tasks | Camera Relocalization, Pose Estimation |
| Published | 2019-11-26 |
| URL | https://arxiv.org/abs/1911.11534v1 |
| https://arxiv.org/pdf/1911.11534v1.pdf | |
| PWC | https://paperswithcode.com/paper/decoupling-features-and-coordinates-for-few |
| Repo | |
| Framework | |
Incremental Intervention Effects in Studies with Many Timepoints, Repeated Outcomes, and Dropout
| Title | Incremental Intervention Effects in Studies with Many Timepoints, Repeated Outcomes, and Dropout |
| Authors | Kwangho Kim, Edward H. Kennedy, Ashley I. Naimi |
| Abstract | Modern longitudinal studies feature data collected at many timepoints, often of the same order of sample size. Such studies are typically affected by dropout and positivity violations. We tackle these problems by generalizing effects of recent incremental interventions (which shift propensity scores rather than set treatment values deterministically) to accommodate multiple outcomes and subject dropout. We give an identifying expression for incremental effects when dropout is conditionally ignorable (without requiring treatment positivity), and derive the nonparametric efficiency bound for estimating such effects. Then we present efficient nonparametric estimators, showing that they converge at fast parametric rates and yield uniform inferential guarantees, even when nuisance functions are estimated flexibly at slower rates. We also study the efficiency of incremental effects relative to more conventional deterministic effects in a novel infinite time horizon setting, where the number of timepoints grows with sample size, and show that incremental effects yield near-exponential gains in this setup. Finally we conclude with simulations and apply our methods in a study of the effect of low-dose aspirin on pregnancy outcomes. |
| Tasks | |
| Published | 2019-07-09 |
| URL | https://arxiv.org/abs/1907.04004v1 |
| https://arxiv.org/pdf/1907.04004v1.pdf | |
| PWC | https://paperswithcode.com/paper/incremental-intervention-effects-in-studies |
| Repo | |
| Framework | |
Stochastic Approximation of Smooth and Strongly Convex Functions: Beyond the $O(1/T)$ Convergence Rate
| Title | Stochastic Approximation of Smooth and Strongly Convex Functions: Beyond the $O(1/T)$ Convergence Rate |
| Authors | Lijun Zhang, Zhi-Hua Zhou |
| Abstract | Stochastic approximation (SA) is a classical approach for stochastic convex optimization. Previous studies have demonstrated that the convergence rate of SA can be improved by introducing either smoothness or strong convexity condition. In this paper, we make use of smoothness and strong convexity simultaneously to boost the convergence rate. Let $\lambda$ be the modulus of strong convexity, $\kappa$ be the condition number, $F_*$ be the minimal risk, and $\alpha>1$ be some small constant. First, we demonstrate that, in expectation, an $O(1/[\lambda T^\alpha] + \kappa F_*/T)$ risk bound is attainable when $T = \Omega(\kappa^\alpha)$. Thus, when $F_*$ is small, the convergence rate could be faster than $O(1/[\lambda T])$ and approaches $O(1/[\lambda T^\alpha])$ in the ideal case. Second, to further benefit from small risk, we show that, in expectation, an $O(1/2^{T/\kappa}+F_*)$ risk bound is achievable. Thus, the excess risk reduces exponentially until reaching $O(F_*)$, and if $F_*=0$, we obtain a global linear convergence. Finally, we emphasize that our proof is constructive and each risk bound is equipped with an efficient stochastic algorithm attaining that bound. |
| Tasks | |
| Published | 2019-01-27 |
| URL | http://arxiv.org/abs/1901.09344v1 |
| http://arxiv.org/pdf/1901.09344v1.pdf | |
| PWC | https://paperswithcode.com/paper/stochastic-approximation-of-smooth-and |
| Repo | |
| Framework | |
Communication Complexity in Locally Private Distribution Estimation and Heavy Hitters
| Title | Communication Complexity in Locally Private Distribution Estimation and Heavy Hitters |
| Authors | Jayadev Acharya, Ziteng Sun |
| Abstract | We consider the problems of distribution estimation and heavy hitter (frequency) estimation under privacy and communication constraints. While these constraints have been studied separately, optimal schemes for one are sub-optimal for the other. We propose a sample-optimal $\varepsilon$-locally differentially private (LDP) scheme for distribution estimation, where each user communicates only one bit, and requires no public randomness. We show that Hadamard Response, a recently proposed scheme for $\varepsilon$-LDP distribution estimation is also utility-optimal for heavy hitter estimation. Finally, we show that unlike distribution estimation, without public randomness where only one bit suffices, any heavy hitter estimation algorithm that communicates $o(\min {\log n, \log k})$ bits from each user cannot be optimal. |
| Tasks | |
| Published | 2019-05-28 |
| URL | https://arxiv.org/abs/1905.11888v1 |
| https://arxiv.org/pdf/1905.11888v1.pdf | |
| PWC | https://paperswithcode.com/paper/communication-complexity-in-locally-private |
| Repo | |
| Framework | |
$(1 + \varepsilon)$-class Classification: an Anomaly Detection Method for Highly Imbalanced or Incomplete Data Sets
| Title | $(1 + \varepsilon)$-class Classification: an Anomaly Detection Method for Highly Imbalanced or Incomplete Data Sets |
| Authors | Maxim Borisyak, Artem Ryzhikov, Andrey Ustyuzhanin, Denis Derkach, Fedor Ratnikov, Olga Mineeva |
| Abstract | Anomaly detection is not an easy problem since distribution of anomalous samples is unknown a priori. We explore a novel method that gives a trade-off possibility between one-class and two-class approaches, and leads to a better performance on anomaly detection problems with small or non-representative anomalous samples. The method is evaluated using several data sets and compared to a set of conventional one-class and two-class approaches. |
| Tasks | Anomaly Detection |
| Published | 2019-06-14 |
| URL | https://arxiv.org/abs/1906.06096v1 |
| https://arxiv.org/pdf/1906.06096v1.pdf | |
| PWC | https://paperswithcode.com/paper/1-varepsilon-class-classification-an-anomaly |
| Repo | |
| Framework | |
Automated email Generation for Targeted Attacks using Natural Language
| Title | Automated email Generation for Targeted Attacks using Natural Language |
| Authors | Avisha Das, Rakesh Verma |
| Abstract | With an increasing number of malicious attacks, the number of people and organizations falling prey to social engineering attacks is proliferating. Despite considerable research in mitigation systems, attackers continually improve their modus operandi by using sophisticated machine learning, natural language processing techniques with an intent to launch successful targeted attacks aimed at deceiving detection mechanisms as well as the victims. We propose a system for advanced email masquerading attacks using Natural Language Generation (NLG) techniques. Using legitimate as well as an influx of varying malicious content, the proposed deep learning system generates \textit{fake} emails with malicious content, customized depending on the attacker’s intent. The system leverages Recurrent Neural Networks (RNNs) for automated text generation. We also focus on the performance of the generated emails in defeating statistical detectors, and compare and analyze the emails using a proposed baseline. |
| Tasks | Text Generation |
| Published | 2019-08-19 |
| URL | https://arxiv.org/abs/1908.06893v1 |
| https://arxiv.org/pdf/1908.06893v1.pdf | |
| PWC | https://paperswithcode.com/paper/automated-email-generation-for-targeted |
| Repo | |
| Framework | |
Predictive Multi-level Patient Representations from Electronic Health Records
| Title | Predictive Multi-level Patient Representations from Electronic Health Records |
| Authors | Zichang Wang, Haoran Li, Luchen Liu, Haoxian Wu, Ming Zhang |
| Abstract | The advent of the Internet era has led to an explosive growth in the Electronic Health Records (EHR) in the past decades. The EHR data can be regarded as a collection of clinical events, including laboratory results, medication records, physiological indicators, etc, which can be used for clinical outcome prediction tasks to support constructions of intelligent health systems. Learning patient representation from these clinical events for the clinical outcome prediction is an important but challenging step. Most related studies transform EHR data of a patient into a sequence of clinical events in temporal order and then use sequential models to learn patient representations for outcome prediction. However, clinical event sequence contains thousands of event types and temporal dependencies. We further make an observation that clinical events occurring in a short period are not constrained by any temporal order but events in a long term are influenced by temporal dependencies. The multi-scale temporal property makes it difficult for traditional sequential models to capture the short-term co-occurrence and the long-term temporal dependencies in clinical event sequences. In response to the above challenges, this paper proposes a Multi-level Representation Model (MRM). MRM first uses a sparse attention mechanism to model the short-term co-occurrence, then uses interval-based event pooling to remove redundant information and reduce sequence length and finally predicts clinical outcomes through Long Short-Term Memory (LSTM). Experiments on real-world datasets indicate that our proposed model largely improves the performance of clinical outcome prediction tasks using EHR data. |
| Tasks | |
| Published | 2019-11-12 |
| URL | https://arxiv.org/abs/1911.05698v1 |
| https://arxiv.org/pdf/1911.05698v1.pdf | |
| PWC | https://paperswithcode.com/paper/predictive-multi-level-patient |
| Repo | |
| Framework | |
Medi-Care AI: Predicting Medications From Billing Codes via Robust Recurrent Neural Networks
| Title | Medi-Care AI: Predicting Medications From Billing Codes via Robust Recurrent Neural Networks |
| Authors | Deyin Liu, Lin Wu, Xue Li |
| Abstract | In this paper, we present an effective deep prediction framework based on robust recurrent neural networks (RNNs) to predict the likely therapeutic classes of medications a patient is taking, given a sequence of diagnostic billing codes in their record. Accurately capturing the list of medications currently taken by a given patient is extremely challenging due to undefined errors and omissions. We present a general robust framework that explicitly models the possible contamination through overtime decay mechanism on the input billing codes and noise injection into the recurrent hidden states, respectively. By doing this, billing codes are reformulated into its temporal patterns with decay rates on each medical variable, and the hidden states of RNNs are regularised by random noises which serve as dropout to improved RNNs robustness towards data variability in terms of missing values and multiple errors. The proposed method is extensively evaluated on real health care data to demonstrate its effectiveness in suggesting medication orders from contaminated values. |
| Tasks | |
| Published | 2019-11-14 |
| URL | https://arxiv.org/abs/2001.10065v1 |
| https://arxiv.org/pdf/2001.10065v1.pdf | |
| PWC | https://paperswithcode.com/paper/medi-care-ai-predicting-medications-from |
| Repo | |
| Framework | |
Predicting Progression of Age-related Macular Degeneration from Fundus Images using Deep Learning
| Title | Predicting Progression of Age-related Macular Degeneration from Fundus Images using Deep Learning |
| Authors | Boris Babenko, Siva Balasubramanian, Katy E. Blumer, Greg S. Corrado, Lily Peng, Dale R. Webster, Naama Hammel, Avinash V. Varadarajan |
| Abstract | Background: Patients with neovascular age-related macular degeneration (AMD) can avoid vision loss via certain therapy. However, methods to predict the progression to neovascular age-related macular degeneration (nvAMD) are lacking. Purpose: To develop and validate a deep learning (DL) algorithm to predict 1-year progression of eyes with no, early, or intermediate AMD to nvAMD, using color fundus photographs (CFP). Design: Development and validation of a DL algorithm. Methods: We trained a DL algorithm to predict 1-year progression to nvAMD, and used 10-fold cross-validation to evaluate this approach on two groups of eyes in the Age-Related Eye Disease Study (AREDS): none/early/intermediate AMD, and intermediate AMD (iAMD) only. We compared the DL algorithm to the manually graded 4-category and 9-step scales in the AREDS dataset. Main outcome measures: Performance of the DL algorithm was evaluated using the sensitivity at 80% specificity for progression to nvAMD. Results: The DL algorithm’s sensitivity for predicting progression to nvAMD from none/early/iAMD (78+/-6%) was higher than manual grades from the 9-step scale (67+/-8%) or the 4-category scale (48+/-3%). For predicting progression specifically from iAMD, the DL algorithm’s sensitivity (57+/-6%) was also higher compared to the 9-step grades (36+/-8%) and the 4-category grades (20+/-0%). Conclusions: Our DL algorithm performed better in predicting progression to nvAMD than manual grades. Future investigations are required to test the application of this DL algorithm in a real-world clinical setting. |
| Tasks | |
| Published | 2019-04-10 |
| URL | http://arxiv.org/abs/1904.05478v1 |
| http://arxiv.org/pdf/1904.05478v1.pdf | |
| PWC | https://paperswithcode.com/paper/predicting-progression-of-age-related-macular |
| Repo | |
| Framework | |
Characterizing the impact of using features extracted from pre-trained models on the quality of video captioning sequence-to-sequence models
| Title | Characterizing the impact of using features extracted from pre-trained models on the quality of video captioning sequence-to-sequence models |
| Authors | Menatallh Hammad, May Hammad, Mohamed Elshenawy |
| Abstract | The task of video captioning, that is, the automatic generation of sentences describing a sequence of actions in a video, has attracted an increasing attention recently. The complex and high-dimensional representation of video data makes it difficult for a typical encoder-decoder architectures to recognize relevant features and encode them in a proper format. Video data contains different modalities that can be recognized using a mix image, scene, action and audio features. In this paper, we characterize the different features affecting video descriptions and explore the interactions among these features and how they affect the final quality of a video representation. Building on existing encoder-decoder models that utilize limited range of video information, our comparisons show how the inclusion of multi-modal video features can make a significant effect on improving the quality of generated statements. The work is of special interest to scientists and practitioners who are using sequence-to-sequence models to generate video captions. |
| Tasks | Video Captioning |
| Published | 2019-11-22 |
| URL | https://arxiv.org/abs/1911.09989v1 |
| https://arxiv.org/pdf/1911.09989v1.pdf | |
| PWC | https://paperswithcode.com/paper/characterizing-the-impact-of-using-features |
| Repo | |
| Framework | |
