Paper Group ANR 178
Device Heterogeneity in Federated Learning: A Superquantile Approach. Two Tier Prediction of Stroke Using Artificial Neural Networks and Support Vector Machines. Unsupervised Pixel-level Road Defect Detection via Adversarial Image-to-Frequency Transform. Backward Feature Correction: How Deep Learning Performs Deep Learning. Spectral Clustering Revi …
Device Heterogeneity in Federated Learning: A Superquantile Approach
| Title | Device Heterogeneity in Federated Learning: A Superquantile Approach |
| Authors | Yassine Laguel, Krishna Pillutla, Jérôme Malick, Zaid Harchaoui |
| Abstract | We propose a federated learning framework to handle heterogeneous client devices which do not conform to the population data distribution. The approach hinges upon a parameterized superquantile-based objective, where the parameter ranges over levels of conformity. We present an optimization algorithm and establish its convergence to a stationary point. We show how to practically implement it using secure aggregation by interleaving iterations of the usual federated averaging method with device filtering. We conclude with numerical experiments on neural networks as well as linear models on tasks from computer vision and natural language processing. |
| Tasks | |
| Published | 2020-02-25 |
| URL | https://arxiv.org/abs/2002.11223v1 |
| https://arxiv.org/pdf/2002.11223v1.pdf | |
| PWC | https://paperswithcode.com/paper/device-heterogeneity-in-federated-learning-a |
| Repo | |
| Framework | |
Two Tier Prediction of Stroke Using Artificial Neural Networks and Support Vector Machines
| Title | Two Tier Prediction of Stroke Using Artificial Neural Networks and Support Vector Machines |
| Authors | Jerrin Thomas Panachakel, Jeena R. S |
| Abstract | Cerebrovascular accident (CVA) or stroke is the rapid loss of brain function due to disturbance in the blood supply to the brain. Statistically, stroke is the second leading cause of death. This has motivated us to suggest a two-tier system for predicting stroke; the first tier makes use of Artificial Neural Network (ANN) to predict the chances of a person suffering from stroke. The ANN is trained the using the values of various risk factors of stroke of several patients who had stroke. Once a person is classified as having a high risk of stroke, s/he undergoes another the tier-2 classification test where his/her neuro MRI (Magnetic resonance imaging) is analysed to predict the chances of stroke. The tier-2 uses Non-negative Matrix Factorization and Haralick Textural features for feature extraction and SVM classifier for classification. We have obtained an accuracy of 96.67% in tier-1 and an accuracy of 70% in tier-2. |
| Tasks | |
| Published | 2020-03-17 |
| URL | https://arxiv.org/abs/2003.08354v2 |
| https://arxiv.org/pdf/2003.08354v2.pdf | |
| PWC | https://paperswithcode.com/paper/two-tier-prediction-of-stroke-using |
| Repo | |
| Framework | |
Unsupervised Pixel-level Road Defect Detection via Adversarial Image-to-Frequency Transform
| Title | Unsupervised Pixel-level Road Defect Detection via Adversarial Image-to-Frequency Transform |
| Authors | Jongmin Yu, Duyong Kim, Younkwan Lee, Moongu Jeon |
| Abstract | In the past few years, the performance of road defect detection has been remarkably improved thanks to advancements on various studies on computer vision and deep learning. Although a large-scale and well-annotated datasets enhance the performance of detecting road pavement defects to some extent, it is still challengeable to derive a model which can perform reliably for various road conditions in practice, because it is intractable to construct a dataset considering diverse road conditions and defect patterns. To end this, we propose an unsupervised approach to detecting road defects, using Adversarial Image-to-Frequency Transform (AIFT). AIFT adopts the unsupervised manner and adversarial learning in deriving the defect detection model, so AIFT does not need annotations for road pavement defects. We evaluate the efficiency of AIFT using GAPs384 dataset, Cracktree200 dataset, CRACK500 dataset, and CFD dataset. The experimental results demonstrate that the proposed approach detects various road detects, and it outperforms existing state-of-the-art approaches. |
| Tasks | |
| Published | 2020-01-30 |
| URL | https://arxiv.org/abs/2001.11175v2 |
| https://arxiv.org/pdf/2001.11175v2.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-pixel-level-road-defect |
| Repo | |
| Framework | |
Backward Feature Correction: How Deep Learning Performs Deep Learning
| Title | Backward Feature Correction: How Deep Learning Performs Deep Learning |
| Authors | Zeyuan Allen-Zhu, Yuanzhi Li |
| Abstract | How does a 110-layer ResNet learn a high-complexity classifier using relatively few training examples and short training time? We present a theory towards explaining this in terms of $\textit{hierarchical learning}$. We refer hierarchical learning as the learner learns to represent a complicated target function by decomposing it into a sequence of simpler functions to reduce sample and time complexity. This paper formally analyzes how multi-layer neural networks can perform such hierarchical learning efficiently and automatically simply by applying stochastic gradient descent (SGD). On the conceptual side, we present, to the best of our knowledge, the FIRST theory result indicating how very deep neural networks can still be sample and time efficient on certain hierarchical learning tasks, when NO KNOWN non-hierarchical algorithms (such as kernel method, linear regression over feature mappings, tensor decomposition, sparse coding) are efficient. We establish a new principle called “backward feature correction”, which we believe is the key to understand the hierarchical learning in multi-layer neural networks. On the technical side, we show for regression and even for binary classification, for every input dimension $d > 0$, there is a concept class consisting of degree $\omega(1)$ multi-variate polynomials so that, using $\omega(1)$-layer neural networks as learners, SGD can learn any target function from this class in $\mathsf{poly}(d)$ time using $\mathsf{poly}(d)$ samples to any $\frac{1}{\mathsf{poly}(d)}$ error, through learning to represent it as a composition of $\omega(1)$ layers of quadratic functions. In contrast, we present lower bounds stating that several non-hierarchical learners, including any kernel methods, neural tangent kernels, must suffer from $d^{\omega(1)}$ sample or time complexity to learn functions in this concept class even to any $d^{-0.01}$ error. |
| Tasks | |
| Published | 2020-01-13 |
| URL | https://arxiv.org/abs/2001.04413v1 |
| https://arxiv.org/pdf/2001.04413v1.pdf | |
| PWC | https://paperswithcode.com/paper/backward-feature-correction-how-deep-learning |
| Repo | |
| Framework | |
Spectral Clustering Revisited: Information Hidden in the Fiedler Vector
| Title | Spectral Clustering Revisited: Information Hidden in the Fiedler Vector |
| Authors | Adela DePavia, Stefan Steinerberger |
| Abstract | We are interested in the clustering problem on graphs: it is known that if there are two underlying clusters, then the signs of the eigenvector corresponding to the second largest eigenvalue of the adjacency matrix can reliably reconstruct the two clusters. We argue that the vertices for which the eigenvector has the largest and the smallest entries, respectively, are unusually strongly connected to their own cluster and more reliably classified than the rest. This can be regarded as a discrete version of the Hot Spots conjecture and should be useful in applications. We give a rigorous proof for the stochastic block model and several examples. |
| Tasks | |
| Published | 2020-03-22 |
| URL | https://arxiv.org/abs/2003.09969v1 |
| https://arxiv.org/pdf/2003.09969v1.pdf | |
| PWC | https://paperswithcode.com/paper/spectral-clustering-revisited-information |
| Repo | |
| Framework | |
Automatic Detection of Coronavirus Disease (COVID-19) Using X-ray Images and Deep Convolutional Neural Networks
| Title | Automatic Detection of Coronavirus Disease (COVID-19) Using X-ray Images and Deep Convolutional Neural Networks |
| Authors | Ali Narin, Ceren Kaya, Ziynet Pamuk |
| Abstract | The 2019 novel coronavirus (COVID-19), with a starting point in China, has spread rapidly among people living in other countries, and is approaching approximately 305,275 cases worldwide according to the statistics of European Centre for Disease Prevention and Control. There are a limited number of COVID-19 test kits available in hospitals due to the increasing cases daily. Therefore, it is necessary to implement an automatic detection system as a quick alternative diagnosis option to prevent COVID-19 spreading among people. In this study, three different convolutional neural network based models (ResNet50, InceptionV3 and Inception-ResNetV2) have been proposed for the detection of coronavirus pneumonia infected patient using chest X-ray radiographs. ROC analyses and confusion matrices by these three models are given and analyzed using 5-fold cross validation. Considering the performance results obtained, it is seen that the pre-trained ResNet50 model provides the highest classification performance with 98% accuracy among other two proposed models (97% accuracy for InceptionV3 and 87% accuracy for Inception-ResNetV2). |
| Tasks | |
| Published | 2020-03-24 |
| URL | https://arxiv.org/abs/2003.10849v1 |
| https://arxiv.org/pdf/2003.10849v1.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-detection-of-coronavirus-disease |
| Repo | |
| Framework | |
Faster Activity and Data Detection in Massive Random Access: A Multi-armed Bandit Approach
| Title | Faster Activity and Data Detection in Massive Random Access: A Multi-armed Bandit Approach |
| Authors | Jialin Dong, Jun Zhang, Yuanming Shi, Jessie Hui Wang |
| Abstract | This paper investigates the grant-free random access with massive IoT devices. By embedding the data symbols in the signature sequences, joint device activity detection and data decoding can be achieved, which, however, significantly increases the computational complexity. Coordinate descent algorithms that enjoy a low per-iteration complexity have been employed to solve the detection problem, but previous works typically employ a random coordinate selection policy which leads to slow convergence. In this paper, we develop multi-armed bandit approaches for more efficient detection via coordinate descent, which make a delicate trade-off between exploration and exploitation in coordinate selection. Specifically, we first propose a bandit based strategy, i.e., Bernoulli sampling, to speed up the convergence rate of coordinate descent, by learning which coordinates will result in more aggressive descent of the objective function. To further improve the convergence rate, an inner multi-armed bandit problem is established to learn the exploration policy of Bernoulli sampling. Both convergence rate analysis and simulation results are provided to show that the proposed bandit based algorithms enjoy faster convergence rates with a lower time complexity compared with the state-of-the-art algorithm. Furthermore, our proposed algorithms are applicable to different scenarios, e.g., massive random access with low-precision analog-to-digital converters (ADCs). |
| Tasks | Action Detection, Activity Detection |
| Published | 2020-01-28 |
| URL | https://arxiv.org/abs/2001.10237v1 |
| https://arxiv.org/pdf/2001.10237v1.pdf | |
| PWC | https://paperswithcode.com/paper/faster-activity-and-data-detection-in-massive |
| Repo | |
| Framework | |
Automatically Searching for U-Net Image Translator Architecture
| Title | Automatically Searching for U-Net Image Translator Architecture |
| Authors | Han Shu, Yunhe Wang |
| Abstract | Image translators have been successfully applied to many important low level image processing tasks. However, classical network architecture of image translator like U-Net, is borrowed from other vision tasks like biomedical image segmentation. This straightforward adaptation may not be optimal and could cause redundancy in the network structure. In this paper, we propose an automatic architecture searching method for image translator. By utilizing evolutionary algorithm, we investigate a more efficient network architecture which costs less computation resources and achieves better performance than the original one. Extensive qualitative and quantitative experiments are conducted to demonstrate the effectiveness of the proposed method. Moreover, we transplant the searched network architecture to other datasets which are not involved in the architecture searching procedure. Efficiency of the searched architecture on these datasets further demonstrates the generalization of the method. |
| Tasks | Semantic Segmentation |
| Published | 2020-02-26 |
| URL | https://arxiv.org/abs/2002.11581v1 |
| https://arxiv.org/pdf/2002.11581v1.pdf | |
| PWC | https://paperswithcode.com/paper/automatically-searching-for-u-net-image |
| Repo | |
| Framework | |
Massively Multilingual Document Alignment with Cross-lingual Sentence-Mover’s Distance
| Title | Massively Multilingual Document Alignment with Cross-lingual Sentence-Mover’s Distance |
| Authors | Ahmed El-Kishky, Francisco Guzmán |
| Abstract | Cross-lingual document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other. Such aligned data can be used for a variety of NLP tasks from training cross-lingual representations to mining parallel bitexts for machine translation training. In this paper we develop an unsupervised scoring function that leverages cross-lingual sentence embeddings to compute the semantic distance between documents in different languages. These semantic distances are then used to guide a document alignment algorithm to properly pair cross-lingual web documents across a variety of low, mid, and high-resource language pairs. Recognizing that our proposed scoring function and other state of the art methods are computationally intractable for long web documents, we utilize a more tractable greedy algorithm that performs comparably. We experimentally demonstrate that our distance metric performs better alignment than current baselines outperforming them by 7% on high-resource language pairs, 15% on mid-resource language pairs, and 22% on low-resource language pairs |
| Tasks | Machine Translation, Sentence Embeddings |
| Published | 2020-01-31 |
| URL | https://arxiv.org/abs/2002.00761v1 |
| https://arxiv.org/pdf/2002.00761v1.pdf | |
| PWC | https://paperswithcode.com/paper/massively-multilingual-document-alignment |
| Repo | |
| Framework | |
Reservoir memory machines
| Title | Reservoir memory machines |
| Authors | Benjamin Paassen, Alexander Schulz |
| Abstract | In recent years, Neural Turing Machines have gathered attention by joining the flexibility of neural networks with the computational capabilities of Turing machines. However, Neural Turing Machines are notoriously hard to train, which limits their applicability. We propose reservoir memory machines, which are still able to solve some of the benchmark tests for Neural Turing Machines, but are much faster to train, requiring only an alignment algorithm and linear regression. Our model can also be seen as an extension of echo state networks with an external memory, enabling arbitrarily long storage without interference. |
| Tasks | |
| Published | 2020-02-12 |
| URL | https://arxiv.org/abs/2003.04793v1 |
| https://arxiv.org/pdf/2003.04793v1.pdf | |
| PWC | https://paperswithcode.com/paper/reservoir-memory-machines |
| Repo | |
| Framework | |
PrivGen: Preserving Privacy of Sequences Through Data Generation
| Title | PrivGen: Preserving Privacy of Sequences Through Data Generation |
| Authors | Sigal Shaked, Lior Rokach |
| Abstract | Sequential data is everywhere, and it can serve as a basis for research that will lead to improved processes. For example, road infrastructure can be improved by identifying bottlenecks in GPS data, or early diagnosis can be improved by analyzing patterns of disease progression in medical data. The main obstacle is that access and use of such data is usually limited or not permitted at all due to concerns about violating user privacy, and rightly so. Anonymizing sequence data is not a simple task, since a user creates an almost unique signature over time. Existing anonymization methods reduce the quality of information in order to maintain the level of anonymity required. Damage to quality may disrupt patterns that appear in the original data and impair the preservation of various characteristics. Since in many cases the researcher does not need the data as is and instead is only interested in the patterns that exist in the data, we propose PrivGen, an innovative method for generating data that maintains patterns and characteristics of the source data. We demonstrate that the data generation mechanism significantly limits the risk of privacy infringement. Evaluating our method with real-world datasets shows that its generated data preserves many characteristics of the data, including the sequential model, as trained based on the source data. This suggests that the data generated by our method could be used in place of actual data for various types of analysis, maintaining user privacy and the data’s integrity at the same time. |
| Tasks | |
| Published | 2020-02-23 |
| URL | https://arxiv.org/abs/2002.09834v1 |
| https://arxiv.org/pdf/2002.09834v1.pdf | |
| PWC | https://paperswithcode.com/paper/privgen-preserving-privacy-of-sequences |
| Repo | |
| Framework | |
Tracking of Micro Unmanned Aerial Vehicles: A Comparative Study
| Title | Tracking of Micro Unmanned Aerial Vehicles: A Comparative Study |
| Authors | Fatih Gökçe |
| Abstract | Micro unmanned aerial vehicles (mUAV) became very common in recent years. As a result of their widespread usage, when they are flown by hobbyists illegally, crucial risks are imposed and such mUAVs need to be sensed by security systems. Furthermore, the sensing of mUAVs are essential for also swarm robotics research where the individuals in a flock of robots require systems to sense and localize each other for coordinated operation. In order to obtain such systems, there are studies to detect mUAVs utilizing different sensing mediums, such as vision, infrared and sound signals, and small-scale radars. However, there are still challenges that awaits to be handled in this field such as integrating tracking approaches to the vision-based detection systems to enhance accuracy and computational complexity. For this reason, in this study, we combine various tracking approaches to a vision-based mUAV detection system available in the literature, in order to evaluate different tracking approaches in terms of accuracy and as well as investigate the effect of such integration to the computational cost. |
| Tasks | |
| Published | 2020-01-16 |
| URL | https://arxiv.org/abs/2001.06066v1 |
| https://arxiv.org/pdf/2001.06066v1.pdf | |
| PWC | https://paperswithcode.com/paper/tracking-of-micro-unmanned-aerial-vehicles-a |
| Repo | |
| Framework | |
VFlow: More Expressive Generative Flows with Variational Data Augmentation
| Title | VFlow: More Expressive Generative Flows with Variational Data Augmentation |
| Authors | Jianfei Chen, Cheng Lu, Biqi Chenli, Jun Zhu, Tian Tian |
| Abstract | Generative flows are promising tractable models for density modeling that define probabilistic distributions with invertible transformations. However, tractability imposes architectural constraints on generative flows, making them less expressive than other types of generative models. In this work, we study a previously overlooked constraint that all the intermediate representations must have the same dimensionality with the original data due to invertibility, limiting the width of the network. We tackle this constraint by augmenting the data with some extra dimensions and jointly learning a generative flow for augmented data as well as the distribution of augmented dimensions under a variational inference framework. Our approach, VFlow, is a generalization of generative flows and therefore always performs better. Combining with existing generative flows, VFlow achieves a new state-of-the-art 2.98 bits per dimension on the CIFAR-10 dataset and is more compact than previous models to reach similar modeling quality. |
| Tasks | Data Augmentation |
| Published | 2020-02-22 |
| URL | https://arxiv.org/abs/2002.09741v1 |
| https://arxiv.org/pdf/2002.09741v1.pdf | |
| PWC | https://paperswithcode.com/paper/vflow-more-expressive-generative-flows-with |
| Repo | |
| Framework | |
Training Question Answering Models From Synthetic Data
| Title | Training Question Answering Models From Synthetic Data |
| Authors | Raul Puri, Ryan Spring, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro |
| Abstract | Question and answer generation is a data augmentation method that aims to improve question answering (QA) models given the limited amount of human labeled data. However, a considerable gap remains between synthetic and human-generated question-answer pairs. This work aims to narrow this gap by taking advantage of large language models and explores several factors such as model size, quality of pretrained models, scale of data synthesized, and algorithmic choices. On the SQuAD1.1 question answering task, we achieve higher accuracy using solely synthetic questions and answers than when using the SQuAD1.1 training set questions alone. Removing access to real Wikipedia data, we synthesize questions and answers from a synthetic corpus generated by an 8.3 billion parameter GPT-2 model. With no access to human supervision and only access to other models, we are able to train state of the art question answering networks on entirely model-generated data that achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. We further apply our methodology to SQuAD2.0 and show a 2.8 absolute gain on EM score compared to prior work using synthetic data. |
| Tasks | Data Augmentation, Question Answering |
| Published | 2020-02-22 |
| URL | https://arxiv.org/abs/2002.09599v1 |
| https://arxiv.org/pdf/2002.09599v1.pdf | |
| PWC | https://paperswithcode.com/paper/training-question-answering-models-from |
| Repo | |
| Framework | |
Low-Budget Label Query through Domain Alignment Enforcement
| Title | Low-Budget Label Query through Domain Alignment Enforcement |
| Authors | Jurandy Almeida, Cristiano Saltori, Paolo Rota, Nicu Sebe |
| Abstract | Deep learning revolution happened thanks to the availability of a massive amount of labelled data which have contributed to the development of models with extraordinary inference capabilities. Despite the public availability of a large quantity of datasets, to address specific requirements it is often necessary to generate a new set of labelled data. Quite often, the production of labels is costly and sometimes it requires specific know-how to be fulfilled. In this work, we tackle a new problem named low-budget label query that consists in suggesting to the user a small (low budget) set of samples to be labelled, from a completely unlabelled dataset, with the final goal of maximizing the classification accuracy on that dataset. In this work we first improve an Unsupervised Domain Adaptation (UDA) method to better align source and target domains using consistency constraints, reaching the state of the art on a few UDA tasks. Finally, using the previously trained model as reference, we propose a simple yet effective selection method based on uniform sampling of the prediction consistency distribution, which is deterministic and steadily outperforms other baselines as well as competing models on a large variety of publicly available datasets. |
| Tasks | Domain Adaptation, Unsupervised Domain Adaptation |
| Published | 2020-01-01 |
| URL | https://arxiv.org/abs/2001.00238v2 |
| https://arxiv.org/pdf/2001.00238v2.pdf | |
| PWC | https://paperswithcode.com/paper/low-budget-unsupervised-label-query-through |
| Repo | |
| Framework | |
