April 2, 2020

3192 words 15 mins read

Paper Group ANR 369

Unsupervised Style and Content Separation by Minimizing Mutual Information for Speech Synthesis. Breast Cancer Detection Using Convolutional Neural Networks. Hybrid Cryptocurrency Pump and Dump Detection. Affinity Graph Supervision for Visual Recognition. Building and Interpreting Deep Similarity Models. Learning to Encode and Classify Test Executi …

Unsupervised Style and Content Separation by Minimizing Mutual Information for Speech Synthesis


Title	Unsupervised Style and Content Separation by Minimizing Mutual Information for Speech Synthesis
Authors	Ting-Yao Hu, Ashish Shrivastava, Oncel Tuzel, Chandra Dhir
Abstract	We present a method to generate speech from input text and a style vector that is extracted from a reference speech signal in an unsupervised manner, i.e., no style annotation, such as speaker information, is required. Existing unsupervised methods, during training, generate speech by computing style from the corresponding ground truth sample and use a decoder to combine the style vector with the input text. Training the model in such a way leaks content information into the style vector. The decoder can use the leaked content and ignore some of the input text to minimize the reconstruction loss. At inference time, when the reference speech does not match the content input, the output may not contain all of the content of the input text. We refer to this problem as “content leakage”, which we address by explicitly estimating and minimizing the mutual information between the style and the content through an adversarial training formulation. We call our method MIST - Mutual Information based Style Content Separation. The main goal of the method is to preserve the input content in the synthesized speech signal, which we measure by the word error rate (WER) and show substantial improvements over state-of-the-art unsupervised speech synthesis methods.
Tasks	Speech Synthesis
Published	2020-03-09
URL	https://arxiv.org/abs/2003.06227v1
PDF	https://arxiv.org/pdf/2003.06227v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-style-and-content-separation-by
Repo
Framework

Breast Cancer Detection Using Convolutional Neural Networks


Title	Breast Cancer Detection Using Convolutional Neural Networks
Authors	Simon Hadush, Yaecob Girmay, Abiot Sinamo, Gebrekirstos Hagos
Abstract	Breast cancer is prevalent in Ethiopia that accounts 34% among women cancer patients. The diagnosis technique in Ethiopia is manual which was proven to be tedious, subjective, and challenging. Deep learning techniques are revolutionizing the field of medical image analysis and hence in this study, we proposed Convolutional Neural Networks (CNNs) for breast mass detection so as to minimize the overheads of manual analysis. CNN architecture is designed for the feature extraction stage and adapted both the Region Proposal Network (RPN) and Region of Interest (ROI) portion of the faster R-CNN for the automated breast mass abnormality detection. Our model detects mass region and classifies them into benign or malignant abnormality in mammogram(MG) images at once. For the proposed model, MG images were collected from different hospitals, locally.The images were passed through different preprocessing stages such as gaussian filter, median filter, bilateral filters and extracted the region of the breast from the background of the MG image. The performance of the model on test dataset is found to be: detection accuracy 91.86%, sensitivity of 94.67% and AUC-ROC of 92.2%.
Tasks	Anomaly Detection, Breast Cancer Detection
Published	2020-03-17
URL	https://arxiv.org/abs/2003.07911v2
PDF	https://arxiv.org/pdf/2003.07911v2.pdf
PWC	https://paperswithcode.com/paper/breast-cancer-detection-using-convolutional
Repo
Framework

Hybrid Cryptocurrency Pump and Dump Detection


Title	Hybrid Cryptocurrency Pump and Dump Detection
Authors	Hadi Mansourifar, Lin Chen, Weidong Shi
Abstract	Increasingly growing Cryptocurrency markets have become a hive for scammers to run pump and dump schemes which is considered as an anomalous activity in exchange markets. Anomaly detection in time series is challenging since existing methods are not sufficient to detect the anomalies in all contexts. In this paper, we propose a novel hybrid pump and dump detection method based on distance and density metrics. First, we propose a novel automatic thresh-old setting method for distance-based anomaly detection. Second, we propose a novel metric called density score for density-based anomaly detection. Finally, we exploit the combination of density and distance metrics successfully as a hybrid approach. Our experiments show that, the proposed hybrid approach is reliable to detect the majority of alleged P & D activities in top ranked exchange pairs by outperforming both density-based and distance-based methods.
Tasks	Anomaly Detection, Time Series
Published	2020-03-14
URL	https://arxiv.org/abs/2003.06551v1
PDF	https://arxiv.org/pdf/2003.06551v1.pdf
PWC	https://paperswithcode.com/paper/hybrid-cryptocurrency-pump-and-dump-detection
Repo
Framework

Affinity Graph Supervision for Visual Recognition


Title	Affinity Graph Supervision for Visual Recognition
Authors	Chu Wang, Babak Samari, Vladimir G. Kim, Siddhartha Chaudhuri, Kaleem Siddiqi
Abstract	Affinity graphs are widely used in deep architectures, including graph convolutional neural networks and attention networks. Thus far, the literature has focused on abstracting features from such graphs, while the learning of the affinities themselves has been overlooked. Here we propose a principled method to directly supervise the learning of weights in affinity graphs, to exploit meaningful connections between entities in the data source. Applied to a visual attention network, our affinity supervision improves relationship recovery between objects, even without the use of manually annotated relationship labels. We further show that affinity learning between objects boosts scene categorization performance and that the supervision of affinity can also be applied to graphs built from mini-batches, for neural network training. In an image classification task we demonstrate consistent improvement over the baseline, with diverse network architectures and datasets.
Tasks	Image Classification
Published	2020-03-19
URL	https://arxiv.org/abs/2003.09049v1
PDF	https://arxiv.org/pdf/2003.09049v1.pdf
PWC	https://paperswithcode.com/paper/affinity-graph-supervision-for-visual
Repo
Framework

Building and Interpreting Deep Similarity Models


Title	Building and Interpreting Deep Similarity Models
Authors	Oliver Eberle, Jochen Büttner, Florian Kräutli, Klaus-Robert Müller, Matteo Valleriani, Grégoire Montavon
Abstract	Many learning algorithms such as kernel machines, nearest neighbors, clustering, or anomaly detection, are based on the concept of ‘distance’ or ‘similarity’. Before similarities are used for training an actual machine learning model, we would like to verify that they are bound to meaningful patterns in the data. In this paper, we propose to make similarities interpretable by augmenting them with an explanation in terms of input features. We develop BiLRP, a scalable and theoretically founded method to systematically decompose similarity scores on pairs of input features. Our method can be expressed as a composition of LRP explanations, which were shown in previous works to scale to highly nonlinear functions. Through an extensive set of experiments, we demonstrate that BiLRP robustly explains complex similarity models, e.g. built on VGG-16 deep neural network features. Additionally, we apply our method to an open problem in digital humanities: detailed assessment of similarity between historical documents such as astronomical tables. Here again, BiLRP provides insight and brings verifiability into a highly engineered and problem-specific similarity model.
Tasks	Anomaly Detection
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05431v1
PDF	https://arxiv.org/pdf/2003.05431v1.pdf
PWC	https://paperswithcode.com/paper/building-and-interpreting-deep-similarity
Repo
Framework

Learning to Encode and Classify Test Executions


Title	Learning to Encode and Classify Test Executions
Authors	Foivos Tsimpourlas, Ajitha Rajan, Miltiadis Allamanis
Abstract	The challenge of automatically determining the correctness of test executions is referred to as the test oracle problem and is one of the key remaining issues for automated testing. The goal in this paper is to solve the test oracle problem in a way that is general, scalable and accurate. To achieve this, we use supervised learning over test execution traces. We label a small fraction of the execution traces with their verdict of pass or fail. We use the labelled traces to train a neural network (NN) model to learn to distinguish runtime patterns for passing versus failing executions for a given program. Our approach for building this NN model involves the following steps, 1. Instrument the program to record execution traces as sequences of method invocations and global state, 2. Label a small fraction of the execution traces with their verdicts, 3. Designing a NN component that embeds information in execution traces to fixed length vectors, 4. Design a NN model that uses the trace information for classification, 5. Evaluate the inferred classification model on unseen execution traces from the program. We evaluate our approach using case studies from different application domains: 1. Module from Ethereum Blockchain, 2. Module from PyTorch deep learning framework, 3. Microsoft SEAL encryption library components, 4. Sed stream editor, 5. Value pointer library and 6. Nine network protocols from Linux packet identifier, L7-Filter. We found the classification models for all subject programs resulted in high precision, recall and specificity, over 95%, while only training with an average 9% of the total traces. Our experiments show that the proposed neural network model is highly effective as a test oracle and is able to learn runtime patterns to distinguish passing and failing test executions for systems and tests from different application domains.
Tasks
Published	2020-01-08
URL	https://arxiv.org/abs/2001.02444v1
PDF	https://arxiv.org/pdf/2001.02444v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-encode-and-classify-test
Repo
Framework

Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing


Title	Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing
Authors	Youngduck Choi, Youngnam Lee, Junghyun Cho, Jineon Baek, Byungsoo Kim, Yeongmin Cha, Dongmin Shin, Chan Bae, Jaewe Heo
Abstract	Knowledge tracing, the act of modeling a student’s knowledge through learning activities, is an extensively studied problem in the field of computer-aided education. Although models with attention mechanism have outperformed traditional approaches such as Bayesian knowledge tracing and collaborative filtering, they share two limitations. Firstly, the models rely on shallow attention layers and fail to capture complex relations among exercises and responses over time. Secondly, different combinations of queries, keys and values for the self-attention layer for knowledge tracing were not extensively explored. Usual practice of using exercises and interactions (exercise-response pairs) as queries and keys/values respectively lacks empirical support. In this paper, we propose a novel Transformer based model for knowledge tracing, SAINT: Separated Self-AttentIve Neural Knowledge Tracing. SAINT has an encoder-decoder structure where exercise and response embedding sequence separately enter the encoder and the decoder respectively, which allows to stack attention layers multiple times. To the best of our knowledge, this is the first work to suggest an encoder-decoder model for knowledge tracing that applies deep self-attentive layers to exercises and responses separately. The empirical evaluations on a large-scale knowledge tracing dataset show that SAINT achieves the state-of-the-art performance in knowledge tracing with the improvement of AUC by 1.8% compared to the current state-of-the-art models.
Tasks	Knowledge Tracing
Published	2020-02-14
URL	https://arxiv.org/abs/2002.07033v1
PDF	https://arxiv.org/pdf/2002.07033v1.pdf
PWC	https://paperswithcode.com/paper/towards-an-appropriate-query-key-and-value
Repo
Framework

Wrapper Feature Selection Algorithm for the Optimization of an Indicator System of Patent Value Assessment


Title	Wrapper Feature Selection Algorithm for the Optimization of an Indicator System of Patent Value Assessment
Authors	Yihui Qiu, Chiyu Zhang
Abstract	Effective patent value assessment provides decision support for patent transection and promotes the practical application of patent technology. The limitations of previous research on patent value assessment were analyzed in this work, and a wrapper-mode feature selection algorithm that is based on classifier prediction accuracy was developed. Verification experiments on multiple UCI standard datasets indicated that the algorithm effectively reduced the size of the feature set and significantly enhanced the prediction accuracy of the classifier. When the algorithm was utilized to establish an indicator system of patent value assessment, the size of the system was reduced, and the generalization performance of the classifier was enhanced. Sequential forward selection was applied to further reduce the size of the indicator set and generate an optimal indicator system of patent value assessment.
Tasks	Feature Selection
Published	2020-01-21
URL	https://arxiv.org/abs/2001.08371v1
PDF	https://arxiv.org/pdf/2001.08371v1.pdf
PWC	https://paperswithcode.com/paper/wrapper-feature-selection-algorithm-for-the
Repo
Framework

The estimation error of general first order methods


Title	The estimation error of general first order methods
Authors	Michael Celentano, Andrea Montanari, Yuchen Wu
Abstract	Modern large-scale statistical models require to estimate thousands to millions of parameters. This is often accomplished by iterative algorithms such as gradient descent, projected gradient descent or their accelerated versions. What are the fundamental limits to these approaches? This question is well understood from an optimization viewpoint when the underlying objective is convex. Work in this area characterizes the gap to global optimality as a function of the number of iterations. However, these results have only indirect implications in terms of the gap to statistical optimality. Here we consider two families of high-dimensional estimation problems: high-dimensional regression and low-rank matrix estimation, and introduce a class of `general first order methods’ that aim at efficiently estimating the underlying parameters. This class of algorithms is broad enough to include classical first order optimization (for convex and non-convex objectives), but also other types of algorithms. Under a random design assumption, we derive lower bounds on the estimation error that hold in the high-dimensional asymptotics in which both the number of observations and the number of parameters diverge. These lower bounds are optimal in the sense that there exist algorithms whose estimation error matches the lower bounds up to asymptotically negligible terms. We illustrate our general results through applications to sparse phase retrieval and sparse principal component analysis. \|
Tasks
Published	2020-02-28
URL	https://arxiv.org/abs/2002.12903v2
PDF	https://arxiv.org/pdf/2002.12903v2.pdf
PWC	https://paperswithcode.com/paper/the-estimation-error-of-general-first-order
Repo
Framework

Nyström Subspace Learning for Large-scale SVMs


Title	Nyström Subspace Learning for Large-scale SVMs
Authors	Weida Li, Mingxia Liu, Daoqiang Zhang
Abstract	As an implementation of the Nystr"{o}m method, Nystr"{o}m computational regularization (NCR) imposed on kernel classification and kernel ridge regression has proven capable of achieving optimal bounds in the large-scale statistical learning setting, while enjoying much better time complexity. In this study, we propose a Nystr"{o}m subspace learning (NSL) framework to reveal that all you need for employing the Nystr"{o}m method, including NCR, upon any kernel SVM is to use the efficient off-the-shelf linear SVM solvers as a black box. Based on our analysis, the bounds developed for the Nystr"{o}m method are linked to NSL, and the analytical difference between two distinct implementations of the Nystr"{o}m method is clearly presented. Besides, NSL also leads to sharper theoretical results for the clustered Nystr"{o}m method. Finally, both regression and classification tasks are performed to compare two implementations of the Nystr"{o}m method.
Tasks
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08937v1
PDF	https://arxiv.org/pdf/2002.08937v1.pdf
PWC	https://paperswithcode.com/paper/nystrom-subspace-learning-for-large-scale
Repo
Framework

Strategy to Increase the Safety of a DNN-based Perception for HAD Systems


Title	Strategy to Increase the Safety of a DNN-based Perception for HAD Systems
Authors	Timo Sämann, Peter Schlicht, Fabian Hüger
Abstract	Safety is one of the most important development goals for highly automated driving (HAD) systems. This applies in particular to the perception function driven by Deep Neural Networks (DNNs). For these, large parts of the traditional safety processes and requirements are not fully applicable or sufficient. The aim of this paper is to present a framework for the description and mitigation of DNN insufficiencies and the derivation of relevant safety mechanisms to increase the safety of DNNs. To assess the effectiveness of these safety mechanisms, we present a categorization scheme for evaluation metrics.
Tasks
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08935v1
PDF	https://arxiv.org/pdf/2002.08935v1.pdf
PWC	https://paperswithcode.com/paper/strategy-to-increase-the-safety-of-a-dnn
Repo
Framework

Neural Network Segmentation of Interstitial Fibrosis, Tubular Atrophy, and Glomerulosclerosis in Renal Biopsies


Title	Neural Network Segmentation of Interstitial Fibrosis, Tubular Atrophy, and Glomerulosclerosis in Renal Biopsies
Authors	Brandon Ginley, Kuang-Yu Jen, Avi Rosenberg, Felicia Yen, Sanjay Jain, Agnes Fogo, Pinaki Sarder
Abstract	Glomerulosclerosis, interstitial fibrosis, and tubular atrophy (IFTA) are histologic indicators of irrecoverable kidney injury. In standard clinical practice, the renal pathologist visually assesses, under the microscope, the percentage of sclerotic glomeruli and the percentage of renal cortical involvement by IFTA. Estimation of IFTA is a subjective process due to a varied spectrum and definition of morphological manifestations. Modern artificial intelligence and computer vision algorithms have the ability to reduce inter-observer variability through rigorous quantitation. In this work, we apply convolutional neural networks for the segmentation of glomerulosclerosis and IFTA in periodic acid-Schiff stained renal biopsies. The convolutional network approach achieves high performance in intra-institutional holdout data, and achieves moderate performance in inter-intuitional holdout data, which the network had never seen in training. The convolutional approach demonstrated interesting properties, such as learning to predict regions better than the provided ground truth as well as developing its own conceptualization of segmental sclerosis. Subsequent estimations of IFTA and glomerulosclerosis percentages showed high correlation with ground truth.
Tasks
Published	2020-02-28
URL	https://arxiv.org/abs/2002.12868v1
PDF	https://arxiv.org/pdf/2002.12868v1.pdf
PWC	https://paperswithcode.com/paper/neural-network-segmentation-of-interstitial
Repo
Framework

Hypergraph Clustering in the Weighted Stochastic Block Model via Convex Relaxation of Truncated MLE


Title	Hypergraph Clustering in the Weighted Stochastic Block Model via Convex Relaxation of Truncated MLE
Authors	Jeonghwan Lee, Daesung Kim, Hye Won Chung
Abstract	We study hypergraph clustering under the weighted $d$-uniform hypergraph stochastic block model ($d$-WHSBM), where each edge consisting of $d$ nodes has higher expected weight if $d$ nodes are from the same community compared to edges consisting of nodes from different communities. We propose a new hypergraph clustering algorithm, which is a convex relaxation of truncated maximum likelihood estimator (CRTMLE), that can handle the relatively sparse, high-dimensional regime of the $d$-WHSBM with community sizes of different orders. We provide performance guarantees of this algorithm under a unified framework for different parameter regimes, and show that it achieves the order-wise optimal or the best existing results for approximately balanced community sizes. We also demonstrate the first recovery guarantees for the setting with growing number of communities of unbalanced sizes.
Tasks
Published	2020-03-23
URL	https://arxiv.org/abs/2003.10038v1
PDF	https://arxiv.org/pdf/2003.10038v1.pdf
PWC	https://paperswithcode.com/paper/hypergraph-clustering-in-the-weighted
Repo
Framework

A memory of motion for visual predictive control tasks


Title	A memory of motion for visual predictive control tasks
Authors	Antonio Paolillo, Teguh Santoso Lembono, Sylvain Calinon
Abstract	This paper addresses the problem of efficiently achieving visual predictive control tasks. To this end, a memory of motion, containing a set of trajectories built off-line, is used for leveraging precomputation and dealing with difficult visual tasks. Standard regression techniques, such as k-nearest neighbors and Gaussian process regression, are used to query the memory and provide on-line a warm-start and a way point to the control optimization process. The proposed technique allows the control scheme to achieve high performance and, at the same time, keep the computational time limited. Simulation and experimental results, carried out with a 7-axis manipulator, show the effectiveness of the approach.
Tasks
Published	2020-01-31
URL	https://arxiv.org/abs/2001.11759v2
PDF	https://arxiv.org/pdf/2001.11759v2.pdf
PWC	https://paperswithcode.com/paper/using-a-memory-of-motion-to-efficiently
Repo
Framework

Brain MRI-based 3D Convolutional Neural Networks for Classification of Schizophrenia and Controls


Title	Brain MRI-based 3D Convolutional Neural Networks for Classification of Schizophrenia and Controls
Authors	Mengjiao Hu, Kang Sim, Juan Helen Zhou, Xudong Jiang, Cuntai Guan
Abstract	Convolutional Neural Network (CNN) has been successfully applied on classification of both natural images and medical images but not yet been applied to differentiating patients with schizophrenia from healthy controls. Given the subtle, mixed, and sparsely distributed brain atrophy patterns of schizophrenia, the capability of automatic feature learning makes CNN a powerful tool for classifying schizophrenia from controls as it removes the subjectivity in selecting relevant spatial features. To examine the feasibility of applying CNN to classification of schizophrenia and controls based on structural Magnetic Resonance Imaging (MRI), we built 3D CNN models with different architectures and compared their performance with a handcrafted feature-based machine learning approach. Support vector machine (SVM) was used as classifier and Voxel-based Morphometry (VBM) was used as feature for handcrafted feature-based machine learning. 3D CNN models with sequential architecture, inception module and residual module were trained from scratch. CNN models achieved higher cross-validation accuracy than handcrafted feature-based machine learning. Moreover, testing on an independent dataset, 3D CNN models greatly outperformed handcrafted feature-based machine learning. This study underscored the potential of CNN for identifying patients with schizophrenia using 3D brain MR images and paved the way for imaging-based individual-level diagnosis and prognosis in psychiatric disorders.
Tasks
Published	2020-03-14
URL	https://arxiv.org/abs/2003.08818v1
PDF	https://arxiv.org/pdf/2003.08818v1.pdf
PWC	https://paperswithcode.com/paper/brain-mri-based-3d-convolutional-neural
Repo
Framework