Paper Group ANR 1231
Abnormal Client Behavior Detection in Federated Learning. Interrupted and cascaded permutation invariant training for speech separation. Mixup-breakdown: a consistency training method for improving generalization of speech separation models. Robust Deep Neural Networks Inspired by Fuzzy Logic. A Reinforcement Learning Approach for the Multichannel …
Abnormal Client Behavior Detection in Federated Learning
| Title | Abnormal Client Behavior Detection in Federated Learning |
| Authors | Suyi Li, Yong Cheng, Yang Liu, Wei Wang, Tianjian Chen |
| Abstract | In federated learning systems, clients are autonomous in that their behaviors are not fully governed by the server. Consequently, a client may intentionally or unintentionally deviate from the prescribed course of federated model training, resulting in abnormal behaviors, such as turning into a malicious attacker or a malfunctioning client. Timely detecting those anomalous clients is therefore critical to minimize their adverse impacts. In this work, we propose to detect anomalous clients at the server side. In particular, we generate low-dimensional surrogates of model weight vectors and use them to perform anomaly detection. We evaluate our solution through experiments on image classification model training over the FEMNIST dataset. Experimental results show that the proposed detection-based approach significantly outperforms the conventional defense-based methods. |
| Tasks | Anomaly Detection, Image Classification |
| Published | 2019-10-22 |
| URL | https://arxiv.org/abs/1910.09933v2 |
| https://arxiv.org/pdf/1910.09933v2.pdf | |
| PWC | https://paperswithcode.com/paper/abnormal-client-behavior-detection-in |
| Repo | |
| Framework | |
Interrupted and cascaded permutation invariant training for speech separation
| Title | Interrupted and cascaded permutation invariant training for speech separation |
| Authors | Gene-Ping Yang, Szu-Lin Wu, Yao-Wen Mao, Hung-yi Lee, Lin-shan Lee |
| Abstract | Permutation Invariant Training (PIT) has long been a stepping stone method for training speech separation model in handling the label ambiguity problem. With PIT selecting the minimum cost label assignments dynamically, very few studies considered the separation problem to be optimizing both the model parameters and the label assignments, but focused on searching for good model architecture and parameters. In this paper, we investigate instead for a given model architecture the various flexible label assignment strategies for training the model, rather than directly using PIT. Surprisingly, we discover a significant performance boost compared to PIT is possible if the model is trained with fixed label assignments and a good set of labels is chosen. With fixed label training cascaded between two sections of PIT, we achieved the state-of-the-art performance on WSJ0-2mix without changing the model architecture at all. |
| Tasks | Speech Separation |
| Published | 2019-10-28 |
| URL | https://arxiv.org/abs/1910.12706v1 |
| https://arxiv.org/pdf/1910.12706v1.pdf | |
| PWC | https://paperswithcode.com/paper/interrupted-and-cascaded-permutation |
| Repo | |
| Framework | |
Mixup-breakdown: a consistency training method for improving generalization of speech separation models
| Title | Mixup-breakdown: a consistency training method for improving generalization of speech separation models |
| Authors | Max W. Y. Lam, Jun Wang, Dan Su, Dong Yu |
| Abstract | Deep-learning based speech separation models confront poor generalization problem that even the state-of-the-art models could abruptly fail when evaluating them in mismatch conditions. To address this problem, we propose an easy-to-implement yet effective consistency based semi-supervised learning (SSL) approach, namely Mixup-Breakdown training (MBT). It learns a teacher model to “breakdown” unlabeled inputs, and the estimated separations are interpolated to produce more useful pseudo “mixup” input-output pairs, on which the consistency regularization could apply for learning a student model. In our experiment, we evaluate MBT under various conditions with ascending degrees of mismatch, including unseen interfering speech, noise, and music, and compare MBT’s generalization capability against state-of-the-art supervised learning and SSL approaches. The result indicates that MBT significantly outperforms several strong baselines with up to 13.77% relative SI-SNRi improvement. Moreover, MBT only adds negligible computational overhead to standard training schemes. |
| Tasks | Speech Separation |
| Published | 2019-10-28 |
| URL | https://arxiv.org/abs/1910.13253v3 |
| https://arxiv.org/pdf/1910.13253v3.pdf | |
| PWC | https://paperswithcode.com/paper/mixup-breakdown-a-consistency-training-method |
| Repo | |
| Framework | |
Robust Deep Neural Networks Inspired by Fuzzy Logic
| Title | Robust Deep Neural Networks Inspired by Fuzzy Logic |
| Authors | Minh Le |
| Abstract | Deep neural networks have achieved impressive performance and become the de-facto standard in many tasks. However, troubling phenomena such as adversarial and fooling examples suggest that the generalization they make is flawed. I argue that among the roots of the phenomena are two geometric properties of common deep learning architectures: their distributed nature and the connectedness of their decision regions. As a remedy, I propose new architectures inspired by fuzzy logic that combine several alternative design elements. Through experiments on MNIST and CIFAR-10, the new models are shown to be more local, better at rejecting noise samples, and more robust against adversarial examples. Ablation analyses reveal behaviors on adversarial examples that cannot be explained by the linearity hypothesis but are consistent with the hypothesis that logic-inspired traits create more robust models. |
| Tasks | |
| Published | 2019-11-20 |
| URL | https://arxiv.org/abs/1911.08635v2 |
| https://arxiv.org/pdf/1911.08635v2.pdf | |
| PWC | https://paperswithcode.com/paper/logic-inspired-deep-neural-networks |
| Repo | |
| Framework | |
A Reinforcement Learning Approach for the Multichannel Rendezvous Problem
| Title | A Reinforcement Learning Approach for the Multichannel Rendezvous Problem |
| Authors | Jen-Hung Wang, Ping-En Lu, Cheng-Shang Chang, Duan-Shin Lee |
| Abstract | In this paper, we consider the multichannel rendezvous problem in cognitive radio networks (CRNs) where the probability that two users hopping on the same channel have a successful rendezvous is a function of channel states. The channel states are modelled by two-state Markov chains that have a good state and a bad state. These channel states are not observable by the users. For such a multichannel rendezvous problem, we are interested in finding the optimal policy to minimize the expected time-to-rendezvous (ETTR) among the class of {\em dynamic blind rendezvous policies}, i.e., at the $t^{th}$ time slot each user selects channel $i$ independently with probability $p_i(t)$, $i=1,2, \ldots, N$. By formulating such a multichannel rendezvous problem as an adversarial bandit problem, we propose using a reinforcement learning approach to learn the channel selection probabilities $p_i(t)$, $i=1,2, \ldots, N$. Our experimental results show that the reinforcement learning approach is very effective and yields comparable ETTRs when comparing to various approximation policies in the literature. |
| Tasks | |
| Published | 2019-07-02 |
| URL | https://arxiv.org/abs/1907.01919v2 |
| https://arxiv.org/pdf/1907.01919v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-reinforcement-learning-approach-for-the |
| Repo | |
| Framework | |
Conditionally Learn to Pay Attention for Sequential Visual Task
| Title | Conditionally Learn to Pay Attention for Sequential Visual Task |
| Authors | Jun He, Quan-Jie Cao, Lei Zhang |
| Abstract | Sequential visual task usually requires to pay attention to its current interested object conditional on its previous observations. Different from popular soft attention mechanism, we propose a new attention framework by introducing a novel conditional global feature which represents the weak feature descriptor of the current focused object. Specifically, for a standard CNN (Convolutional Neural Network) pipeline, the convolutional layers with different receptive fields are used to produce the attention maps by measuring how the convolutional features align to the conditional global feature. The conditional global feature can be generated by different recurrent structure according to different visual tasks, such as a simple recurrent neural network for multiple objects recognition, or a moderate complex language model for image caption. Experiments show that our proposed conditional attention model achieves the best performance on the SVHN (Street View House Numbers) dataset with / without extra bounding box; and for image caption, our attention model generates better scores than the popular soft attention model. |
| Tasks | Language Modelling |
| Published | 2019-11-11 |
| URL | https://arxiv.org/abs/1911.04365v1 |
| https://arxiv.org/pdf/1911.04365v1.pdf | |
| PWC | https://paperswithcode.com/paper/conditionally-learn-to-pay-attention-for |
| Repo | |
| Framework | |
COCO_TS Dataset: Pixel-level Annotations Based on Weak Supervision for Scene Text Segmentation
| Title | COCO_TS Dataset: Pixel-level Annotations Based on Weak Supervision for Scene Text Segmentation |
| Authors | Simone Bonechi, Paolo Andreini, Monica Bianchini, Franco Scarselli |
| Abstract | The absence of large scale datasets with pixel-level supervisions is a significant obstacle for the training of deep convolutional networks for scene text segmentation. For this reason, synthetic data generation is normally employed to enlarge the training dataset. Nonetheless, synthetic data cannot reproduce the complexity and variability of natural images. In this paper, a weakly supervised learning approach is used to reduce the shift between training on real and synthetic data. Pixel-level supervisions for a text detection dataset (i.e. where only bounding-box annotations are available) are generated. In particular, the COCO-Text-Segmentation (COCO_TS) dataset, which provides pixel-level supervisions for the COCO-Text dataset, is created and released. The generated annotations are used to train a deep convolutional neural network for semantic segmentation. Experiments show that the proposed dataset can be used instead of synthetic data, allowing us to use only a fraction of the training samples and significantly improving the performances. |
| Tasks | Semantic Segmentation, Synthetic Data Generation |
| Published | 2019-04-01 |
| URL | https://arxiv.org/abs/1904.00818v6 |
| https://arxiv.org/pdf/1904.00818v6.pdf | |
| PWC | https://paperswithcode.com/paper/coco_ts-dataset-pixel-level-annotations-based |
| Repo | |
| Framework | |
Learning deep forest with multi-scale Local Binary Pattern features for face anti-spoofing
| Title | Learning deep forest with multi-scale Local Binary Pattern features for face anti-spoofing |
| Authors | Rizhao Cai, Changsheng Chen |
| Abstract | Face Anti-Spoofing (FAS) is significant for the security of face recognition systems. Convolutional Neural Networks (CNNs) have been introduced to the field of the FAS and have achieved competitive performance. However, CNN-based methods are vulnerable to the adversarial attack. Attackers could generate adversarial-spoofing examples to circumvent a CNN-based face liveness detector. Studies about the transferability of the adversarial attack reveal that utilizing handcrafted feature-based methods could improve security in a system-level. Therefore, handcrafted feature-based methods are worth our exploration. In this paper, we introduce the deep forest, which is proposed as an alternative towards CNNs by Zhou et al., in the problem of the FAS. To the best of our knowledge, this is the first attempt at exploiting the deep forest in the problem of FAS. Moreover, we propose to re-devise the representation constructing by using LBP descriptors rather than the Grained-Scanning Mechanism in the original scheme. Our method achieves competitive results. On the benchmark database IDIAP REPLAY-ATTACK, 0% Equal Error Rate (EER) is achieved. This work provides a competitive option in a fusing scheme for improving system-level security and offers important ideas to those who want to explore methods besides CNNs. |
| Tasks | Adversarial Attack, Face Anti-Spoofing, Face Recognition |
| Published | 2019-10-09 |
| URL | https://arxiv.org/abs/1910.03850v1 |
| https://arxiv.org/pdf/1910.03850v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-deep-forest-with-multi-scale-local |
| Repo | |
| Framework | |
Yet another but more efficient black-box adversarial attack: tiling and evolution strategies
| Title | Yet another but more efficient black-box adversarial attack: tiling and evolution strategies |
| Authors | Laurent Meunier, Jamal Atif, Olivier Teytaud |
| Abstract | We introduce a new black-box attack achieving state of the art performances. Our approach is based on a new objective function, borrowing ideas from $\ell_\infty$-white box attacks, and particularly designed to fit derivative-free optimization requirements. It only requires to have access to the logits of the classifier without any other information which is a more realistic scenario. Not only we introduce a new objective function, we extend previous works on black box adversarial attacks to a larger spectrum of evolution strategies and other derivative-free optimization methods. We also highlight a new intriguing property that deep neural networks are not robust to single shot tiled attacks. Our models achieve, with a budget limited to $10,000$ queries, results up to $99.2%$ of success rate against InceptionV3 classifier with $630$ queries to the network on average in the untargeted attacks setting, which is an improvement by $90$ queries of the current state of the art. In the targeted setting, we are able to reach, with a limited budget of $100,000$, $100%$ of success rate with a budget of $6,662$ queries on average, i.e. we need $800$ queries less than the current state of the art. |
| Tasks | Adversarial Attack |
| Published | 2019-10-05 |
| URL | https://arxiv.org/abs/1910.02244v2 |
| https://arxiv.org/pdf/1910.02244v2.pdf | |
| PWC | https://paperswithcode.com/paper/yet-another-but-more-efficient-black-box-1 |
| Repo | |
| Framework | |
Sim-to-Real Domain Adaptation For High Energy Physics
| Title | Sim-to-Real Domain Adaptation For High Energy Physics |
| Authors | Marouen Baalouch, Maxime Defurne, Jean-Philippe Poli, Noëlie Cherrier |
| Abstract | Particle physics or High Energy Physics (HEP) studies the elementary constituents of matter and their interactions with each other. Machine Learning (ML) has played an important role in HEP analysis and has proven extremely successful in this area. Usually, the ML algorithms are trained on numerical simulations of the experimental setup and then applied to the real experimental data. However, any discrepancy between the simulation and real data may lead to dramatic consequences concerning the performances of the algorithm on real data. In this paper, we present an application of domain adaptation using a Domain Adversarial Neural Network trained on public HEP data. We demonstrate the success of this approach to achieve sim-to-real transfer and ensure the consistency of the ML algorithms performances on real and simulated HEP datasets. |
| Tasks | Domain Adaptation |
| Published | 2019-12-17 |
| URL | https://arxiv.org/abs/1912.08001v1 |
| https://arxiv.org/pdf/1912.08001v1.pdf | |
| PWC | https://paperswithcode.com/paper/sim-to-real-domain-adaptation-for-high-energy |
| Repo | |
| Framework | |
AutoML using Metadata Language Embeddings
| Title | AutoML using Metadata Language Embeddings |
| Authors | Iddo Drori, Lu Liu, Yi Nian, Sharath C. Koorathota, Jie S. Li, Antonio Khalil Moretti, Juliana Freire, Madeleine Udell |
| Abstract | As a human choosing a supervised learning algorithm, it is natural to begin by reading a text description of the dataset and documentation for the algorithms you might use. We demonstrate that the same idea improves the performance of automated machine learning methods. We use language embeddings from modern NLP to improve state-of-the-art AutoML systems by augmenting their recommendations with vector embeddings of datasets and of algorithms. We use these embeddings in a neural architecture to learn the distance between best-performing pipelines. The resulting (meta-)AutoML framework improves on the performance of existing AutoML frameworks. Our zero-shot AutoML system using dataset metadata embeddings provides good solutions instantaneously, running in under one second of computation. Performance is competitive with AutoML systems OBOE, AutoSklearn, AlphaD3M, and TPOT when each framework is allocated a minute of computation. We make our data, models, and code publicly available. |
| Tasks | AutoML |
| Published | 2019-10-08 |
| URL | https://arxiv.org/abs/1910.03698v1 |
| https://arxiv.org/pdf/1910.03698v1.pdf | |
| PWC | https://paperswithcode.com/paper/automl-using-metadata-language-embeddings |
| Repo | |
| Framework | |
Perturbations are not Enough: Generating Adversarial Examples with Spatial Distortions
| Title | Perturbations are not Enough: Generating Adversarial Examples with Spatial Distortions |
| Authors | He Zhao, Trung Le, Paul Montague, Olivier De Vel, Tamas Abraham, Dinh Phung |
| Abstract | Deep neural network image classifiers are reported to be susceptible to adversarial evasion attacks, which use carefully crafted images created to mislead a classifier. Recently, various kinds of adversarial attack methods have been proposed, most of which focus on adding small perturbations to input images. Despite the success of existing approaches, the way to generate realistic adversarial images with small perturbations remains a challenging problem. In this paper, we aim to address this problem by proposing a novel adversarial method, which generates adversarial examples by imposing not only perturbations but also spatial distortions on input images, including scaling, rotation, shear, and translation. As humans are less susceptible to small spatial distortions, the proposed approach can produce visually more realistic attacks with smaller perturbations, able to deceive classifiers without affecting human predictions. We learn our method by amortized techniques with neural networks and generate adversarial examples efficiently by a forward pass of the networks. Extensive experiments on attacking different types of non-robustified classifiers and robust classifiers with defence show that our method has state-of-the-art performance in comparison with advanced attack parallels. |
| Tasks | Adversarial Attack |
| Published | 2019-10-03 |
| URL | https://arxiv.org/abs/1910.01329v1 |
| https://arxiv.org/pdf/1910.01329v1.pdf | |
| PWC | https://paperswithcode.com/paper/perturbations-are-not-enough-generating-1 |
| Repo | |
| Framework | |
Private Federated Learning with Domain Adaptation
| Title | Private Federated Learning with Domain Adaptation |
| Authors | Daniel Peterson, Pallika Kanani, Virendra J. Marathe |
| Abstract | Federated Learning (FL) is a distributed machine learning (ML) paradigm that enables multiple parties to jointly re-train a shared model without sharing their data with any other parties, offering advantages in both scale and privacy. We propose a framework to augment this collaborative model-building with per-user domain adaptation. We show that this technique improves model accuracy for all users, using both real and synthetic data, and that this improvement is much more pronounced when differential privacy bounds are imposed on the FL model. |
| Tasks | Domain Adaptation |
| Published | 2019-12-13 |
| URL | https://arxiv.org/abs/1912.06733v1 |
| https://arxiv.org/pdf/1912.06733v1.pdf | |
| PWC | https://paperswithcode.com/paper/private-federated-learning-with-domain |
| Repo | |
| Framework | |
Open-plan Glare Evaluator (OGE): A New Glare Prediction Model for Open-Plan Offices Using Machine Learning Algorithms
| Title | Open-plan Glare Evaluator (OGE): A New Glare Prediction Model for Open-Plan Offices Using Machine Learning Algorithms |
| Authors | Ayman Wagdy, Veronica Garcia-Hansen, Mohammed Elhenawy, Gillian Isoardi, Robin Drogemuller, Fatma Fathy |
| Abstract | Predicting discomfort glare in open-plan offices is a challenging problem since most of available glare metrics are developed for cellular offices which are typically daylight dominated. The problem with open-plan offices is that they are mainly dependent on electric lighting rather than daylight even when they have a fully glazed facade. In addition, the contrast between bright windows and the buildings interior can be problematic and may cause discomfort glare to the building occupants. These problems can affect occupant productivity and wellbeing. Thus, it is important to develop a predictive model to avoid discomfort glare when designing open plan offices. To the best of our knowledge, we are the first to adopt Machine Learning (ML) models to predict discomfort glare. In order to develop new glare predictive models for these types of offices, Post-Occupancy Evaluation (POE) and High Dynamic Range (HDR) images were collected from 80 occupants (n=80) in four different open-plan offices. Consequently, various multi-region luminance values, luminance and glare indices were calculated and used as input features to train ML models. The accuracy of the ML model was compared to the accuracy of 24 indices which were also evaluated using a Receiver Operating Characteristic (ROC) analysis to identify the best cutoff values (thresholds) for each index for open-plan configurations. Results showed that the ML glare model could predict glare in open-plan offices with an accuracy of 83.8% (0.80 true positive rate and 0.86 true negative rate) which outperformed the accuracy of the previously developed glare metrics. |
| Tasks | |
| Published | 2019-10-12 |
| URL | https://arxiv.org/abs/1910.05594v1 |
| https://arxiv.org/pdf/1910.05594v1.pdf | |
| PWC | https://paperswithcode.com/paper/open-plan-glare-evaluator-oge-a-new-glare |
| Repo | |
| Framework | |
Restoring Images with Unknown Degradation Factors by Recurrent Use of a Multi-branch Network
| Title | Restoring Images with Unknown Degradation Factors by Recurrent Use of a Multi-branch Network |
| Authors | Xing Liu, Masanori Suganuma, Xiyang Luo, Takayuki Okatani |
| Abstract | The employment of convolutional neural networks has achieved unprecedented performance in the task of image restoration for a variety of degradation factors. However, high-performance networks have been specifically designed for a single degradation factor. In this paper, we tackle a harder problem, restoring a clean image from its degraded version with an unknown degradation factor, subject to the condition that it is one of the known factors. Toward this end, we design a network having multiple pairs of input and output branches and use it in a recurrent fashion such that a different branch pair is used at each of the recurrent paths. We reinforce the shared part of the network with improved components so that it can handle different degradation factors. We also propose a two-step training method for the network, which consists of multi-task learning and finetuning. The experimental results show that the proposed network yields at least comparable or sometimes even better performance on four degradation factors as compared with the best dedicated network for each of the four. We also test it on a further harder task where the input image contains multiple degradation factors that are mixed with unknown mixture ratios, showing that it achieves better performance than the previous state-of-the-art method designed for the task. |
| Tasks | Deblurring, Image Restoration, JPEG Artifact Removal, Multi-Task Learning |
| Published | 2019-07-10 |
| URL | https://arxiv.org/abs/1907.04508v2 |
| https://arxiv.org/pdf/1907.04508v2.pdf | |
| PWC | https://paperswithcode.com/paper/joint-learning-of-multiple-image-restoration |
| Repo | |
| Framework | |
