Paper Group ANR 209
Unsupervised Out-of-Distribution Detection with Batch Normalization. FPScreen: A Rapid Similarity Search Tool for Massive Molecular Library Based on Molecular Fingerprint Comparison. Accuracy Improvement of Neural Network Training using Particle Swarm Optimization and its Stability Analysis for Classification. Experiments in Detecting Persuasion Te …
Unsupervised Out-of-Distribution Detection with Batch Normalization
| Title | Unsupervised Out-of-Distribution Detection with Batch Normalization |
| Authors | Jiaming Song, Yang Song, Stefano Ermon |
| Abstract | Likelihood from a generative model is a natural statistic for detecting out-of-distribution (OoD) samples. However, generative models have been shown to assign higher likelihood to OoD samples compared to ones from the training distribution, preventing simple threshold-based detection rules. We demonstrate that OoD detection fails even when using more sophisticated statistics based on the likelihoods of individual samples. To address these issues, we propose a new method that leverages batch normalization. We argue that batch normalization for generative models challenges the traditional i.i.d. data assumption and changes the corresponding maximum likelihood objective. Based on this insight, we propose to exploit in-batch dependencies for OoD detection. Empirical results suggest that this leads to more robust detection for high-dimensional images. |
| Tasks | Out-of-Distribution Detection |
| Published | 2019-10-21 |
| URL | https://arxiv.org/abs/1910.09115v1 |
| https://arxiv.org/pdf/1910.09115v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-out-of-distribution-detection |
| Repo | |
| Framework | |
FPScreen: A Rapid Similarity Search Tool for Massive Molecular Library Based on Molecular Fingerprint Comparison
| Title | FPScreen: A Rapid Similarity Search Tool for Massive Molecular Library Based on Molecular Fingerprint Comparison |
| Authors | Lijun Wang, Jianbing Gong, Yingxia Zhang, Tianmou Liu, Junhui Gao |
| Abstract | We designed a fast similarity search engine for large molecular libraries: FPScreen. We downloaded 100 million molecules’ structure files in PubChem with SDF extension, then applied a computational chemistry tool RDKit to convert each structure file into one line of text in MACCS format and stored them in a text file as our molecule library. The similarity search engine compares the similarity while traversing the 166-bit strings in the library file line by line. FPScreen can complete similarity search through 100 million entries in our molecule library within one hour. That is very fast as a biology computation tool. Additionally, we divided our library into several strides for parallel processing. FPScreen was developed in WEB mode. |
| Tasks | |
| Published | 2019-06-13 |
| URL | https://arxiv.org/abs/1906.06170v1 |
| https://arxiv.org/pdf/1906.06170v1.pdf | |
| PWC | https://paperswithcode.com/paper/fpscreen-a-rapid-similarity-search-tool-for |
| Repo | |
| Framework | |
Accuracy Improvement of Neural Network Training using Particle Swarm Optimization and its Stability Analysis for Classification
| Title | Accuracy Improvement of Neural Network Training using Particle Swarm Optimization and its Stability Analysis for Classification |
| Authors | Arijit Nandi, Nanda Dulal Jana |
| Abstract | Supervised classification is the most active and emerging research trends in today’s scenario. In this view, Artificial Neural Network (ANN) techniques have been widely employed and growing interest to the researchers day by day. ANN training aims to find the proper setting of parameters such as weights ($\textbf{W}$) and biases ($b$) to properly classify the given data samples. The training process is formulated in an error minimization problem which consists of many local optima in the search landscape. In this paper, an enhanced Particle Swarm Optimization is proposed to minimize the error function for classifying real-life data sets. A stability analysis is performed to establish the efficiency of the proposed method for improving classification accuracy. The performance measurement such as confusion matrix, $F$-measure and convergence graph indicates the significant improvement in the classification accuracy. |
| Tasks | |
| Published | 2019-05-11 |
| URL | https://arxiv.org/abs/1905.04522v2 |
| https://arxiv.org/pdf/1905.04522v2.pdf | |
| PWC | https://paperswithcode.com/paper/accuracy-improvement-of-neural-network |
| Repo | |
| Framework | |
Experiments in Detecting Persuasion Techniques in the News
| Title | Experiments in Detecting Persuasion Techniques in the News |
| Authors | Seunghak Yu, Giovanni Da San Martino, Preslav Nakov |
| Abstract | Many recent political events, like the 2016 US Presidential elections or the 2018 Brazilian elections have raised the attention of institutions and of the general public on the role of Internet and social media in influencing the outcome of these events. We argue that a safe democracy is one in which citizens have tools to make them aware of propaganda campaigns. We propose a novel task: performing fine-grained analysis of texts by detecting all fragments that contain propaganda techniques as well as their type. We further design a novel multi-granularity neural network, and we show that it outperforms several strong BERT-based baselines. |
| Tasks | |
| Published | 2019-11-15 |
| URL | https://arxiv.org/abs/1911.06815v1 |
| https://arxiv.org/pdf/1911.06815v1.pdf | |
| PWC | https://paperswithcode.com/paper/experiments-in-detecting-persuasion |
| Repo | |
| Framework | |
On Coresets for Regularized Loss Minimization
| Title | On Coresets for Regularized Loss Minimization |
| Authors | Ryan R. Curtin, Sungjin Im, Ben Moseley, Kirk Pruhs, Alireza Samadian |
| Abstract | We design and mathematically analyze sampling-based algorithms for regularized loss minimization problems that are implementable in popular computational models for large data, in which the access to the data is restricted in some way. Our main result is that if the regularizer’s effect does not become negligible as the norm of the hypothesis scales, and as the data scales, then a uniform sample of modest size is with high probability a coreset. In the case that the loss function is either logistic regression or soft-margin support vector machines, and the regularizer is one of the common recommended choices, this result implies that a uniform sample of size $O(d \sqrt{n})$ is with high probability a coreset of $n$ points in $\Re^d$. We contrast this upper bound with two lower bounds. The first lower bound shows that our analysis of uniform sampling is tight; that is, a smaller uniform sample will likely not be a core set. The second lower bound shows that in some sense uniform sampling is close to optimal, as significantly smaller core sets do not generally exist. |
| Tasks | |
| Published | 2019-05-26 |
| URL | https://arxiv.org/abs/1905.10845v2 |
| https://arxiv.org/pdf/1905.10845v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-coresets-for-regularized-loss-minimization |
| Repo | |
| Framework | |
Input complexity and out-of-distribution detection with likelihood-based generative models
| Title | Input complexity and out-of-distribution detection with likelihood-based generative models |
| Authors | Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F. Núñez, Jordi Luque |
| Abstract | Likelihood-based generative models are a promising resource to detect out-of-distribution (OOD) inputs which could compromise the robustness or reliability of a machine learning system. However, likelihoods derived from such models have been shown to be problematic for detecting certain types of inputs that significantly differ from training data. In this paper, we pose that this problem is due to the excessive influence that input complexity has in generative models’ likelihoods. We report a set of experiments supporting this hypothesis, and use an estimate of input complexity to derive an efficient and parameter-free OOD score, which can be seen as a likelihood-ratio, akin to Bayesian model comparison. We find such score to perform comparably to, or even better than, existing OOD detection approaches under a wide range of data sets, models, model sizes, and complexity estimates. |
| Tasks | Out-of-Distribution Detection |
| Published | 2019-09-25 |
| URL | https://arxiv.org/abs/1909.11480v3 |
| https://arxiv.org/pdf/1909.11480v3.pdf | |
| PWC | https://paperswithcode.com/paper/input-complexity-and-out-of-distribution |
| Repo | |
| Framework | |
Deep interval prediction model with gradient descend optimization method for short-term wind power prediction
| Title | Deep interval prediction model with gradient descend optimization method for short-term wind power prediction |
| Authors | Chaoshun Li, Geng Tang, Xiaoming Xue, Xinbiao Chen, Ruoheng Wang, Chu Zhang |
| Abstract | The application of wind power interval prediction for power systems attempts to give more comprehensive support to dispatchers and operators of the grid. Lower upper bound estimation (LUBE) method is widely applied in interval prediction. However, the existing LUBE approaches are trained by meta-heuristic optimization, which is either time-consuming or show poor effect when the LUBE model is complex. In this paper, a deep interval prediction method is designed in the framework of LUBE and an efficient gradient descend (GD) training approach is proposed to train the LUBE model. In this method, the long short-term memory is selected as a representative to show the modelling approach. The architecture of the proposed model consists of three parts, namely the long short-term memory module, the fully connected layers and the rank ordered module. Two loss functions are specially designed for implementing the GD training method based on the root mean square back propagation algorithm. To verify the performance of the proposed model, conventional LUBE models, as well as popular statistic interval prediction models are compared in numerical experiments. The results show that the proposed approach performs best in terms of effectiveness and efficiency with average 45% promotion in quality of prediction interval and 66% reduction of time consumptions compared to traditional LUBE models. |
| Tasks | |
| Published | 2019-11-19 |
| URL | https://arxiv.org/abs/1911.08160v1 |
| https://arxiv.org/pdf/1911.08160v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-interval-prediction-model-with-gradient |
| Repo | |
| Framework | |
Robust Image Segmentation Quality Assessment without Ground Truth
| Title | Robust Image Segmentation Quality Assessment without Ground Truth |
| Authors | Leixin Zhou, Wenxiang Deng, Xiaodong Wu |
| Abstract | Deep learning based image segmentation methods have achieved great success, even having human-level accuracy in some applications. However, due to the black box nature of deep learning, the best method may fail in some situations. Thus predicting segmentation quality without ground truth would be very crucial especially in clinical practice. Recently, people proposed to train neural networks to estimate the quality score by regression. Although it can achieve promising prediction accuracy, the network suffers robustness problem, e.g. it is vulnerable to adversarial attacks. In this paper, we propose to alleviate this problem by utilizing the difference between the input image and the reconstructed image, which is reconstructed from the segmentation to be assessed. The deep learning based reconstruction network (REC-Net) is trained with the input image masked by the ground truth segmentation against the original input image as the target. The rationale behind is that the trained REC-Net can best reconstruct the input image masked by accurate segmentation. The quality score regression network (REG-Net) is then trained with difference images and the corresponding segmentations as input. In this way, the regression network may have lower chance to overfit to the undesired image features from the original input image, and thus is more robust. Results on ACDC17 dataset demonstrated our method is promising. |
| Tasks | Semantic Segmentation |
| Published | 2019-03-20 |
| URL | http://arxiv.org/abs/1903.08773v1 |
| http://arxiv.org/pdf/1903.08773v1.pdf | |
| PWC | https://paperswithcode.com/paper/robust-image-segmentation-quality-assessment |
| Repo | |
| Framework | |
Document Embeddings vs. Keyphrases vs. Terms: An Online Evaluation in Digital Library Recommender Systems
| Title | Document Embeddings vs. Keyphrases vs. Terms: An Online Evaluation in Digital Library Recommender Systems |
| Authors | Andrew Collins, Joeran Beel |
| Abstract | Many recommendation algorithms are available to digital library recommender system operators. The effectiveness of algorithms is largely unreported by way of online evaluation. We compare a standard term-based recommendation approach to two promising approaches for related-article recommendation in digital libraries: document embeddings, and keyphrases. We evaluate the consistency of their performance across multiple scenarios. Through our recommender-as-a-service Mr. DLib, we delivered 33.5M recommendations to users of Sowiport and Jabref over the course of 19 months, from March 2017 to October 2018. The effectiveness of the algorithms differs significantly between Sowiport and Jabref (Wilcoxon rank-sum test; p < 0.05). There is a ~400% difference in effectiveness between the best and worst algorithm in both scenarios separately. The best performing algorithm in Sowiport (terms) is the worst performing in Jabref. The best performing algorithm in Jabref (keyphrases) is 70% worse in Sowiport, than Sowiport`s best algorithm (click-through rate; 0.1% terms, 0.03% keyphrases). | |
| Tasks | Recommendation Systems |
| Published | 2019-05-27 |
| URL | https://arxiv.org/abs/1905.11244v1 |
| https://arxiv.org/pdf/1905.11244v1.pdf | |
| PWC | https://paperswithcode.com/paper/document-embeddings-vs-keyphrases-vs-terms-an |
| Repo | |
| Framework | |
Isotropic Maximization Loss and Entropic Score: Fast, Accurate, Scalable, Unexposed, Turnkey, and Native Neural Networks Out-of-Distribution Detection
| Title | Isotropic Maximization Loss and Entropic Score: Fast, Accurate, Scalable, Unexposed, Turnkey, and Native Neural Networks Out-of-Distribution Detection |
| Authors | David Macêdo, Tsang Ing Ren, Cleber Zanchettin, Adriano L. I. Oliveira, Alain Tapp, Teresa Ludermir |
| Abstract | Current out-of-distribution detection (ODD) approaches require cumbersome procedures that add undesired side-effects to the solution. In this paper, we argue that the uncertainty in neural networks is mainly due to SoftMax loss anisotropy. Consequently, we propose an isotropic loss (IsoMax) and a decision score (Entropic Score) to significantly improve the ODD performance while keeping the overall solution fast, accurate, scalable, unexposed, turnkey, and native. Our experiments indeed showed that uncertainty is extremely reduced simply by replacing the SoftMax loss without relying on techniques such as adversarial training/validation, special-purpose data augmentation, outlier exposure, ensembles methods, Bayesian mechanisms, generative approaches, metric learning, or additional classifiers/regressions. The results also showed that our straightforward proposal overcomes ODIN, ACET, and is competitive against the Mahalanobis approach besides avoiding their undesired requirements and weaknesses. Since IsoMax loss works as a direct and transparent SoftMax loss drop-in replacement, these techniques may be used combined with our loss to increase the overall performance even more if their associated drawbacks are not a concern in a particular use case. |
| Tasks | Data Augmentation, Metric Learning, Out-of-Distribution Detection |
| Published | 2019-08-15 |
| URL | https://arxiv.org/abs/1908.05569v5 |
| https://arxiv.org/pdf/1908.05569v5.pdf | |
| PWC | https://paperswithcode.com/paper/distinction-maximization-loss-fast-scalable |
| Repo | |
| Framework | |
Node Alertness-Detecting changes in rapidly evolving graphs
| Title | Node Alertness-Detecting changes in rapidly evolving graphs |
| Authors | Mirco A. Mannucci, Deborah Tylor |
| Abstract | In this article we describe a new approach for detecting changes in rapidly evolving large-scale graphs. The key notion involved is local alertness: nodes monitor change within their neighborhoods at each time step. Here we propose a financial local alertness application for cointegrated stock pairs |
| Tasks | |
| Published | 2019-07-02 |
| URL | https://arxiv.org/abs/1907.11623v1 |
| https://arxiv.org/pdf/1907.11623v1.pdf | |
| PWC | https://paperswithcode.com/paper/node-alertness-detecting-changes-in-rapidly |
| Repo | |
| Framework | |
Enhancing Decision Tree based Interpretation of Deep Neural Networks through L1-Orthogonal Regularization
| Title | Enhancing Decision Tree based Interpretation of Deep Neural Networks through L1-Orthogonal Regularization |
| Authors | Nina Schaaf, Marco F. Huber, Johannes Maucher |
| Abstract | One obstacle that so far prevents the introduction of machine learning models primarily in critical areas is the lack of explainability. In this work, a practicable approach of gaining explainability of deep artificial neural networks (NN) using an interpretable surrogate model based on decision trees is presented. Simply fitting a decision tree to a trained NN usually leads to unsatisfactory results in terms of accuracy and fidelity. Using L1-orthogonal regularization during training, however, preserves the accuracy of the NN, while it can be closely approximated by small decision trees. Tests with different data sets confirm that L1-orthogonal regularization yields models of lower complexity and at the same time higher fidelity compared to other regularizers. |
| Tasks | |
| Published | 2019-04-10 |
| URL | https://arxiv.org/abs/1904.05394v2 |
| https://arxiv.org/pdf/1904.05394v2.pdf | |
| PWC | https://paperswithcode.com/paper/enhancing-decision-tree-based-interpretation |
| Repo | |
| Framework | |
Out-of-Distribution Detection Using Neural Rendering Generative Models
| Title | Out-of-Distribution Detection Using Neural Rendering Generative Models |
| Authors | Yujia Huang, Sihui Dai, Tan Nguyen, Richard G. Baraniuk, Anima Anandkumar |
| Abstract | Out-of-distribution (OoD) detection is a natural downstream task for deep generative models, due to their ability to learn the input probability distribution. There are mainly two classes of approaches for OoD detection using deep generative models, viz., based on likelihood measure and the reconstruction loss. However, both approaches are unable to carry out OoD detection effectively, especially when the OoD samples have smaller variance than the training samples. For instance, both flow based and VAE models assign higher likelihood to images from SVHN when trained on CIFAR-10 images. We use a recently proposed generative model known as neural rendering model (NRM) and derive metrics for OoD. We show that NRM unifies both approaches since it provides a likelihood estimate and also carries out reconstruction in each layer of the neural network. Among various measures, we found the joint likelihood of latent variables to be the most effective one for OoD detection. Our results show that when trained on CIFAR-10, lower likelihood (of latent variables) is assigned to SVHN images. Additionally, we show that this metric is consistent across other OoD datasets. To the best of our knowledge, this is the first work to show consistently lower likelihood for OoD data with smaller variance with deep generative models. |
| Tasks | Out-of-Distribution Detection |
| Published | 2019-07-10 |
| URL | https://arxiv.org/abs/1907.04572v1 |
| https://arxiv.org/pdf/1907.04572v1.pdf | |
| PWC | https://paperswithcode.com/paper/out-of-distribution-detection-using-neural |
| Repo | |
| Framework | |
GDRQ: Group-based Distribution Reshaping for Quantization
| Title | GDRQ: Group-based Distribution Reshaping for Quantization |
| Authors | Haibao Yu, Tuopu Wen, Guangliang Cheng, Jiankai Sun, Qi Han, Jianping Shi |
| Abstract | Low-bit quantization is challenging to maintain high performance with limited model capacity (e.g., 4-bit for both weights and activations). Naturally, the distribution of both weights and activations in deep neural network are Gaussian-like. Nevertheless, due to the limited bitwidth of low-bit model, uniform-like distributed weights and activations have been proved to be more friendly to quantization while preserving accuracy~\cite{Han2015Learning}. Motivated by this, we propose Scale-Clip, a Distribution Reshaping technique that can reshape weights or activations into a uniform-like distribution in a dynamic manner. Furthermore, to increase the model capability for a low-bit model, a novel Group-based Quantization algorithm is proposed to split the filters into several groups. Different groups can learn different quantization parameters, which can be elegantly merged in to batch normalization layer without extra computational cost in the inference stage. Finally, we integrate Scale-Clip technique with Group-based Quantization algorithm and propose the Group-based Distribution Reshaping Quantization (GDQR) framework to further improve the quantization performance. Experiments on various networks (e.g. VGGNet and ResNet) and vision tasks (e.g. classification, detection and segmentation) demonstrate that our framework achieves good performance. |
| Tasks | Quantization |
| Published | 2019-08-05 |
| URL | https://arxiv.org/abs/1908.01477v1 |
| https://arxiv.org/pdf/1908.01477v1.pdf | |
| PWC | https://paperswithcode.com/paper/gdrq-group-based-distribution-reshaping-for |
| Repo | |
| Framework | |
Algorithms and Improved bounds for online learning under finite hypothesis class
| Title | Algorithms and Improved bounds for online learning under finite hypothesis class |
| Authors | Ankit Sharma, Late C. A. Murthy |
| Abstract | Online learning is the process of answering a sequence of questions based on the correct answers to the previous questions. It is studied in many research areas such as game theory, information theory and machine learning. There are two main components of online learning framework. First, the learning algorithm also known as the learner and second, the hypothesis class which is essentially a set of functions which learner uses to predict answers to the questions. Sometimes, this class contains some functions which have the capability to provide correct answers to the entire sequence of questions. This case is called realizable case. And when hypothesis class does not contain such functions is called unrealizable case. The goal of the learner, in both the cases, is to make as few mistakes as that could have been made by most powerful functions in hypothesis class over the entire sequence of questions. Performance of the learners is analysed by theoretical bounds on the number of mistakes made by them. This paper proposes three algorithms to improve the mistakes bound in the unrealizable case. Proposed algorithms perform highly better than the existing ones in the long run when most of the input sequences presented to the learner are likely to be realizable. |
| Tasks | |
| Published | 2019-03-24 |
| URL | http://arxiv.org/abs/1903.10870v1 |
| http://arxiv.org/pdf/1903.10870v1.pdf | |
| PWC | https://paperswithcode.com/paper/algorithms-and-improved-bounds-for-online |
| Repo | |
| Framework | |
