April 2, 2020

2943 words 14 mins read

Paper Group ANR 228

Shared Task: Lexical Semantic Change Detection in German. Learning to Detect Malicious Clients for Robust Federated Learning. Evaluating Registration Without Ground Truth. Identifying Audio Adversarial Examples via Anomalous Pattern Detection. Iterative energy-based projection on a normal data manifold for anomaly localization. Batch Normalization …

Shared Task: Lexical Semantic Change Detection in German


Title	Shared Task: Lexical Semantic Change Detection in German
Authors	Adnan Ahmad, Kiflom Desta, Fabian Lang, Dominik Schlechtweg
Abstract	Recent NLP architectures have illustrated in various ways how semantic change can be captured across time and domains. However, in terms of evaluation there is a lack of benchmarks to compare the performance of these systems against each other. We present the results of the first shared task on unsupervised lexical semantic change detection (LSCD) in German based on the evaluation framework proposed by Schlechtweg et al. (2019).
Tasks
Published	2020-01-21
URL	https://arxiv.org/abs/2001.07786v1
PDF	https://arxiv.org/pdf/2001.07786v1.pdf
PWC	https://paperswithcode.com/paper/shared-task-lexical-semantic-change-detection
Repo
Framework

Learning to Detect Malicious Clients for Robust Federated Learning


Title	Learning to Detect Malicious Clients for Robust Federated Learning
Authors	Suyi Li, Yong Cheng, Wei Wang, Yang Liu, Tianjian Chen
Abstract	Federated learning systems are vulnerable to attacks from malicious clients. As the central server in the system cannot govern the behaviors of the clients, a rogue client may initiate an attack by sending malicious model updates to the server, so as to degrade the learning performance or enforce targeted model poisoning attacks (a.k.a. backdoor attacks). Therefore, timely detecting these malicious model updates and the underlying attackers becomes critically important. In this work, we propose a new framework for robust federated learning where the central server learns to detect and remove the malicious model updates using a powerful detection model, leading to targeted defense. We evaluate our solution in both image classification and sentiment analysis tasks with a variety of machine learning models. Experimental results show that our solution ensures robust federated learning that is resilient to both the Byzantine attacks and the targeted model poisoning attacks.
Tasks	Image Classification, Sentiment Analysis
Published	2020-02-01
URL	https://arxiv.org/abs/2002.00211v1
PDF	https://arxiv.org/pdf/2002.00211v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-detect-malicious-clients-for
Repo
Framework

Evaluating Registration Without Ground Truth


Title	Evaluating Registration Without Ground Truth
Authors	Carole J. Twining, Vladimir S. Petrović, Timothy F. Cootes, Roy S. Schestowitz, William R. Crum, Christopher J. Taylor
Abstract	We present a generic method for assessing the quality of non-rigid registration (NRR) algorithms, that does not depend on the existence of any ground truth, but depends solely on the data itself. The data is a set of images. The output of any NRR of such a set of images is a dense correspondence across the whole set. Given such a dense correspondence, it is possible to build various generative statistical models of appearance variation across the set. We show that evaluating the quality of the registration can be mapped to the problem of evaluating the quality of the resultant statistical model. The quality of the model entails a comparison between the model and the image data that was used to construct it. It should be noted that this approach does not depend on the specifics of the registration algorithm used (i.e., whether a groupwise or pairwise algorithm was used to register the set of images), or on the specifics of the modelling approach used. We derive an index of image model specificity that can be used to assess image model quality, and hence the quality of registration. This approach is validated by comparing our assessment of registration quality with that derived from ground truth anatomical labeling. We demonstrate that our approach is capable of assessing NRR reliably without ground truth. Finally, to demonstrate the practicality of our method, different NRR algorithms – both pairwise and groupwise – are compared in terms of their performance on 3D MR brain data.
Tasks
Published	2020-02-24
URL	https://arxiv.org/abs/2002.10534v1
PDF	https://arxiv.org/pdf/2002.10534v1.pdf
PWC	https://paperswithcode.com/paper/evaluating-registration-without-ground-truth
Repo
Framework

Identifying Audio Adversarial Examples via Anomalous Pattern Detection


Title	Identifying Audio Adversarial Examples via Anomalous Pattern Detection
Authors	Victor Akinwande, Celia Cintas, Skyler Speakman, Srihari Sridharan
Abstract	Audio processing models based on deep neural networks are susceptible to adversarial attacks even when the adversarial audio waveform is 99.9% similar to a benign sample. Given the wide application of DNN-based audio recognition systems, detecting the presence of adversarial examples is of high practical relevance. By applying anomalous pattern detection techniques in the activation space of these models, we show that 2 of the recent and current state-of-the-art adversarial attacks on audio processing systems systematically lead to higher-than-expected activation at some subset of nodes and we can detect these with up to an AUC of 0.98 with no degradation in performance on benign samples.
Tasks
Published	2020-02-13
URL	https://arxiv.org/abs/2002.05463v1
PDF	https://arxiv.org/pdf/2002.05463v1.pdf
PWC	https://paperswithcode.com/paper/identifying-audio-adversarial-examples-via
Repo
Framework

Iterative energy-based projection on a normal data manifold for anomaly localization


Title	Iterative energy-based projection on a normal data manifold for anomaly localization
Authors	David Dehaene, Oriel Frigo, Sébastien Combrexelle, Pierre Eline
Abstract	Autoencoder reconstructions are widely used for the task of unsupervised anomaly localization. Indeed, an autoencoder trained on normal data is expected to only be able to reconstruct normal features of the data, allowing the segmentation of anomalous pixels in an image via a simple comparison between the image and its autoencoder reconstruction. In practice however, local defects added to a normal image can deteriorate the whole reconstruction, making this segmentation challenging. To tackle the issue, we propose in this paper a new approach for projecting anomalous data on a autoencoder-learned normal data manifold, by using gradient descent on an energy derived from the autoencoder’s loss function. This energy can be augmented with regularization terms that model priors on what constitutes the user-defined optimal projection. By iteratively updating the input of the autoencoder, we bypass the loss of high-frequency information caused by the autoencoder bottleneck. This allows to produce images of higher quality than classic reconstructions. Our method achieves state-of-the-art results on various anomaly localization datasets. It also shows promising results at an inpainting task on the CelebA dataset.
Tasks
Published	2020-02-10
URL	https://arxiv.org/abs/2002.03734v1
PDF	https://arxiv.org/pdf/2002.03734v1.pdf
PWC	https://paperswithcode.com/paper/iterative-energy-based-projection-on-a-normal-1
Repo
Framework

Batch Normalization Biases Deep Residual Networks Towards Shallow Paths


Title	Batch Normalization Biases Deep Residual Networks Towards Shallow Paths
Authors	Soham De, Samuel L. Smith
Abstract	Batch normalization has multiple benefits. It improves the conditioning of the loss landscape, and is a surprisingly effective regularizer. However, the most important benefit of batch normalization arises in residual networks, where it dramatically increases the largest trainable depth. We identify the origin of this benefit: At initialization, batch normalization downscales the residual branch relative to the skip connection, by a normalizing factor proportional to the square root of the network depth. This ensures that, early in training, the function computed by deep normalized residual networks is dominated by shallow paths with well-behaved gradients. We use this insight to develop a simple initialization scheme which can train very deep residual networks without normalization. We also clarify that, although batch normalization does enable stable training with larger learning rates, this benefit is only useful when one wishes to parallelize training over large batch sizes. Our results help isolate the distinct benefits of batch normalization in different architectures.
Tasks
Published	2020-02-24
URL	https://arxiv.org/abs/2002.10444v1
PDF	https://arxiv.org/pdf/2002.10444v1.pdf
PWC	https://paperswithcode.com/paper/batch-normalization-biases-deep-residual
Repo
Framework

Ground Texture Based Localization Using Compact Binary Descriptors


Title	Ground Texture Based Localization Using Compact Binary Descriptors
Authors	Jan Fabian Schmid, Stephan F. Simon, Rudolf Mester
Abstract	Ground texture based localization is a promising approach to achieve high-accuracy positioning of vehicles. We present a self-contained method that can be used for global localization as well as for subsequent local localization updates, i.e. it allows a robot to localize without any knowledge of its current whereabouts, but it can also take advantage of a prior pose estimate to reduce computation time significantly. Our method is based on a novel matching strategy, which we call identity matching, that is based on compact binary feature descriptors. Identity matching treats pairs of features as matches only if their descriptors are identical. While other methods for global localization are faster to compute, our method reaches higher localization success rates, and can switch to local localization after the initial localization.
Tasks
Published	2020-02-25
URL	https://arxiv.org/abs/2002.11061v1
PDF	https://arxiv.org/pdf/2002.11061v1.pdf
PWC	https://paperswithcode.com/paper/ground-texture-based-localization-using
Repo
Framework

Towards Coding for Human and Machine Vision: A Scalable Image Coding Approach


Title	Towards Coding for Human and Machine Vision: A Scalable Image Coding Approach
Authors	Yueyu Hu, Shuai Yang, Wenhan Yang, Ling-Yu Duan, Jiaying Liu
Abstract	The past decades have witnessed the rapid development of image and video coding techniques in the era of big data. However, the signal fidelity-driven coding pipeline design limits the capability of the existing image/video coding frameworks to fulfill the needs of both machine and human vision. In this paper, we come up with a novel image coding framework by leveraging both the compressive and the generative models, to support machine vision and human perception tasks jointly. Given an input image, the feature analysis is first applied, and then the generative model is employed to perform image reconstruction with features and additional reference pixels, in which compact edge maps are extracted in this work to connect both kinds of vision in a scalable way. The compact edge map serves as the basic layer for machine vision tasks, and the reference pixels act as a sort of enhanced layer to guarantee signal fidelity for human vision. By introducing advanced generative models, we train a flexible network to reconstruct images from compact feature representations and the reference pixels. Experimental results demonstrate the superiority of our framework in both human visual quality and facial landmark detection, which provide useful evidence on the emerging standardization efforts on MPEG VCM (Video Coding for Machine).
Tasks	Facial Landmark Detection, Image Reconstruction
Published	2020-01-09
URL	https://arxiv.org/abs/2001.02915v2
PDF	https://arxiv.org/pdf/2001.02915v2.pdf
PWC	https://paperswithcode.com/paper/towards-coding-for-human-and-machine-vision-a
Repo
Framework

Pixel-In-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild


Title	Pixel-In-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild
Authors	Haibo Jin, Shengcai Liao, Ling Shao
Abstract	Recently, heatmap regression based models become popular because of their superior performance on locating facial landmarks. However, high-resolution feature maps have to be either generated repeatedly or maintained through the network for such models, which is computationally inefficient for practical applications. Moreover, their generalization capabilities across domains are rarely explored. To address these two problems, we propose Pixel-In-Pixel (PIP) Net for facial landmark detection. The proposed model is equipped with a novel detection head based on heatmap regression. Different from conventional heatmap regression, the new detection head conducts score prediction on low-resolution feature maps. To localize landmarks more precisely, it also conduct offset predictions within each heatmap pixel. By doing this, the inference time is largely reduced without losing accuracy. Besides, we also propose to leverage unlabeled images to improve the generalization capbility of our model through image translation based data distillation. Extensive experiments on four benchmarks show that PIP Net is comparable to state-of-the-arts while running at $27.8$ FPS on a CPU.
Tasks	Facial Landmark Detection
Published	2020-03-08
URL	https://arxiv.org/abs/2003.03771v1
PDF	https://arxiv.org/pdf/2003.03771v1.pdf
PWC	https://paperswithcode.com/paper/pixel-in-pixel-net-towards-efficient-facial
Repo
Framework

Learning Optimal Temperature Region for Solving Mixed Integer Functional DCOPs


Title	Learning Optimal Temperature Region for Solving Mixed Integer Functional DCOPs
Authors	Saaduddin Mahmud, Md. Mosaddek Khan, Moumita Choudhury, Long Tran-Thanh, Nicholas R. Jennings
Abstract	Distributed Constraint Optimization Problems (DCOPs) are an important framework that models coordinated decision-making problem in multi-agent systems with a set of discrete variables. Later work has extended this to model problems with a set of continuous variables (F-DCOPs). In this paper, we combine both of these models into the Mixed Integer Functional DCOP (MIF-DCOP) model that can deal with problems regardless of its variables’ type. We then propose a novel algorithm, called Distributed Parallel Simulated Annealing (DPSA), where agents cooperatively learn the optimal parameter configuration for the algorithm while also solving the given problem using the learned knowledge. Finally, we empirically benchmark our approach in DCOP, F-DCOP and MIF-DCOP settings and show that DPSA produces solutions of significantly better quality than the state-of-the-art non-exact algorithms in their corresponding setting.
Tasks	Decision Making
Published	2020-02-27
URL	https://arxiv.org/abs/2002.12001v1
PDF	https://arxiv.org/pdf/2002.12001v1.pdf
PWC	https://paperswithcode.com/paper/learning-optimal-temperature-region-for
Repo
Framework

Artificial Intelligence Aided Next-Generation Networks Relying on UAVs


Title	Artificial Intelligence Aided Next-Generation Networks Relying on UAVs
Authors	Xiao Liu, Mingzhe Chen, Yuanwei Liu, Yue Chen, Shuguang Cui, Lajos Hanzo
Abstract	Artificial intelligence (AI) assisted unmanned aerial vehicle (UAV) aided next-generation networking is proposed for dynamic environments. In the AI-enabled UAV-aided wireless networks (UAWN), multiple UAVs are employed as aerial base stations, which are capable of rapidly adapting to the dynamic environment by collecting information about the users’ position and tele-traffic demands, learning from the environment and acting upon the feedback received from the users. Moreover, AI enables the interaction amongst a swarm of UAVs for cooperative optimization of the system. As a benefit of the AI framework, several challenges of conventional UAWN may be circumvented, leading to enhanced network performance, improved reliability and agile adaptivity. As a further benefit, dynamic trajectory design and resource allocation are demonstrated. Finally, potential research challenges and opportunities are discussed.
Tasks
Published	2020-01-28
URL	https://arxiv.org/abs/2001.11958v1
PDF	https://arxiv.org/pdf/2001.11958v1.pdf
PWC	https://paperswithcode.com/paper/artificial-intelligence-aided-next-generation
Repo
Framework

Optimal Multiple Stopping Rule for Warm-Starting Sequential Selection


Title	Optimal Multiple Stopping Rule for Warm-Starting Sequential Selection
Authors	Mathilde Fekom, Nicolas Vayatis, Argyris Kalogeratos
Abstract	In this paper we present the Warm-starting Dynamic Thresholding algorithm, developed using dynamic programming, for a variant of the standard online selection problem. The problem allows job positions to be either free or already occupied at the beginning of the process. Throughout the selection process, the decision maker interviews one after the other the new candidates and reveals a quality score for each of them. Based on that information, she can (re)assign each job at most once by taking immediate and irrevocable decisions. We relax the hard requirement of the class of dynamic programming algorithms to perfectly know the distribution from which the scores of candidates are drawn, by presenting extensions for the partial and no-information cases, in which the decision maker can learn the underlying score distribution sequentially while interviewing candidates.
Tasks
Published	2020-02-12
URL	https://arxiv.org/abs/2002.05160v1
PDF	https://arxiv.org/pdf/2002.05160v1.pdf
PWC	https://paperswithcode.com/paper/optimal-multiple-stopping-rule-for-warm
Repo
Framework

V2I Connectivity-Based Dynamic Queue-Jumper Lane for Emergency Vehicles: An Approximate Dynamic Programming Approach


Title	V2I Connectivity-Based Dynamic Queue-Jumper Lane for Emergency Vehicles: An Approximate Dynamic Programming Approach
Authors	Haoran Su, Joseph Y. J. Chow, Li Jin
Abstract	Emergency vehicle (EV) service is a key function of cities and is exceedingly challenging due to urban traffic congestion. A key contributor to EV service delay is the lack of communication and cooperation between vehicles blocking EVs. In this paper, we study the improvement of EV service using vehicle-to-vehicle connectivity. We consider the establishment of dynamic queue jumper lanes (DQJLs) based on real-time coordination of connected vehicles. We develop a novel stochastic dynamic programming formulation for the DQJL problem, which explicitly account for the uncertainty of drivers’ reaction to approaching EVs. We propose a deep neural network-based approximate dynamic programming (ADP) algorithm that efficiently computes the optimal coordination instructions. We also validate our approach on a micro-simulation testbed using Simulation On Urban Mobility (SUMO).
Tasks
Published	2020-03-02
URL	https://arxiv.org/abs/2003.01025v2
PDF	https://arxiv.org/pdf/2003.01025v2.pdf
PWC	https://paperswithcode.com/paper/v2i-connectivity-based-dynamic-queue-jumper
Repo
Framework

Empirical Study on Airline Delay Analysis and Prediction


Title	Empirical Study on Airline Delay Analysis and Prediction
Authors	Ripon Patgiri, Sajid Hussain, Aditya Nongmeikapam
Abstract	The Big Data analytics are a logical analysis of very large scale datasets. The data analysis enhances an organization and improve the decision making process. In this article, we present Airline Delay Analysis and Prediction to analyze airline datasets with the combination of weather dataset. In this research work, we consider various attributes to analyze flight delay, for example, day-wise, airline-wise, cloud cover, temperature, etc. Moreover, we present rigorous experiments on various machine learning model to predict correctly the delay of a flight, namely, logistic regression with L2 regularization, Gaussian Naive Bayes, K-Nearest Neighbors, Decision Tree classifier and Random forest model. The accuracy of the Random Forest model is 82% with a delay threshold of 15 minutes of flight delay. The analysis is carried out using dataset from 1987 to 2008, the training is conducted with dataset from 2000 to 2007 and validated prediction result using 2008 data. Moreover, we have got recall 99% in the Random Forest model.
Tasks	Decision Making, L2 Regularization
Published	2020-02-17
URL	https://arxiv.org/abs/2002.10254v1
PDF	https://arxiv.org/pdf/2002.10254v1.pdf
PWC	https://paperswithcode.com/paper/empirical-study-on-airline-delay-analysis-and
Repo
Framework

Self-Distillation Amplifies Regularization in Hilbert Space


Title	Self-Distillation Amplifies Regularization in Hilbert Space
Authors	Hossein Mobahi, Mehrdad Farajtabar, Peter L. Bartlett
Abstract	Knowledge distillation introduced in the deep learning context is a method to transfer knowledge from one architecture to another. In particular, when the architectures are identical, this is called self-distillation. The idea is to feed in predictions of the trained model as new target values for retraining (and iterate this loop possibly a few times). It has been empirically observed that the self-distilled model often achieves higher accuracy on held out data. Why this happens, however, has been a mystery: the self-distillation dynamics does not receive any new information about the task and solely evolves by looping over training. To the best of our knowledge, there is no rigorous understanding of why this happens. This work provides the first theoretical analysis of self-distillation. We focus on fitting a nonlinear function to training data, where the model space is Hilbert space and fitting is subject to L2 regularization in this function space. We show that self-distillation iterations modify regularization by progressively limiting the number of basis functions that can be used to represent the solution. This implies (as we also verify empirically) that while a few rounds of self-distillation may reduce over-fitting, further rounds may lead to under-fitting and thus worse performance.
Tasks	L2 Regularization
Published	2020-02-13
URL	https://arxiv.org/abs/2002.05715v2
PDF	https://arxiv.org/pdf/2002.05715v2.pdf
PWC	https://paperswithcode.com/paper/self-distillation-amplifies-regularization-in
Repo
Framework