April 2, 2020

3434 words 17 mins read

Paper Group ANR 109

PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models. Towards Sharper First-Order Adversary with Quantized Gradients. Simulating Performance of ML Systems with Offline Profiling. On a minimum enclosing ball of a collection of linear subspaces. Linear Regression without Correspondences via Concave Minimization. AOL: Adaptiv …

PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models


Title	PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models
Authors	Yinjun Wu, Val Tannen, Susan B. Davidson
Abstract	The ubiquitous use of machine learning algorithms brings new challenges to traditional database problems such as incremental view update. Much effort is being put in better understanding and debugging machine learning models, as well as in identifying and repairing errors in training datasets. Our focus is on how to assist these activities when they have to retrain the machine learning model after removing problematic training samples in cleaning or selecting different subsets of training data for interpretability. This paper presents an efficient provenance-based approach, PrIU, and its optimized version, PrIU-opt, for incrementally updating model parameters without sacrificing prediction accuracy. We prove the correctness and convergence of the incrementally updated model parameters, and validate it experimentally. Experimental results show that up to two orders of magnitude speed-ups can be achieved by PrIU-opt compared to simply retraining the model from scratch, yet obtaining highly similar models.
Tasks
Published	2020-02-26
URL	https://arxiv.org/abs/2002.11791v1
PDF	https://arxiv.org/pdf/2002.11791v1.pdf
PWC	https://paperswithcode.com/paper/priu-a-provenance-based-approach-for
Repo
Framework

Towards Sharper First-Order Adversary with Quantized Gradients


Title	Towards Sharper First-Order Adversary with Quantized Gradients
Authors	Zhuanghua Liu, Ivor W. Tsang
Abstract	Despite the huge success of Deep Neural Networks (DNNs) in a wide spectrum of machine learning and data mining tasks, recent research shows that this powerful tool is susceptible to maliciously crafted adversarial examples. Up until now, adversarial training has been the most successful defense against adversarial attacks. To increase adversarial robustness, a DNN can be trained with a combination of benign and adversarial examples generated by first-order methods. However, in state-of-the-art first-order attacks, adversarial examples with sign gradients retain the sign information of each gradient component but discard the relative magnitude between components. In this work, we replace sign gradients with quantized gradients. Gradient quantization not only preserves the sign information, but also keeps the relative magnitude between components. Experiments show white-box first-order attacks with quantized gradients outperform their variants with sign gradients on multiple datasets. Notably, our BLOB_QG attack achieves an accuracy of $88.32%$ on the secret MNIST model from the MNIST Challenge and it outperforms all other methods on the leaderboard of white-box attacks.
Tasks	Quantization
Published	2020-02-01
URL	https://arxiv.org/abs/2002.02372v1
PDF	https://arxiv.org/pdf/2002.02372v1.pdf
PWC	https://paperswithcode.com/paper/towards-sharper-first-order-adversary-with
Repo
Framework

Simulating Performance of ML Systems with Offline Profiling


Title	Simulating Performance of ML Systems with Offline Profiling
Authors	Hongming Huang, Peng Cheng, Hong Xu, Yongqiang Xiong
Abstract	We advocate that simulation based on offline profiling is a promising approach to better understand and improve the complex ML systems. Our approach uses operation-level profiling and dataflow based simulation to ensure it offers a unified and automated solution for all frameworks and ML models, and is also accurate by considering the various parallelization strategies in a real system.
Tasks
Published	2020-02-17
URL	https://arxiv.org/abs/2002.06790v1
PDF	https://arxiv.org/pdf/2002.06790v1.pdf
PWC	https://paperswithcode.com/paper/simulating-performance-of-ml-systems-with
Repo
Framework

On a minimum enclosing ball of a collection of linear subspaces


Title	On a minimum enclosing ball of a collection of linear subspaces
Authors	Timothy Marrinan, P. -A. Absil, Nicolas Gillis
Abstract	This paper concerns the minimax center of a collection of linear subspaces. When the subspaces are $k$-dimensional subspaces of $\mathbb{R}^n$, this can be cast as finding the center of a minimum enclosing ball on a Grassmann manifold, Gr$(k,n)$. For subspaces of different dimension, the setting becomes a disjoint union of Grassmannians rather than a single manifold, and the problem is no longer well-defined. However, natural geometric maps exist between these manifolds with a well-defined notion of distance for the images of the subspaces under the mappings. Solving the initial problem in this context leads to a candidate minimax center on each of the constituent manifolds, but does not inherently provide intuition about which candidate is the best representation of the data. Additionally, the solutions of different rank are generally not nested so a deflationary approach will not suffice, and the problem must be solved independently on each manifold. We propose and solve an optimization problem parametrized by the rank of the minimax center. The solution is computed using a subgradient algorithm on the dual. By scaling the objective and penalizing the information lost by the rank-$k$ minimax center, we jointly recover an optimal dimension, $k^$, and a central subspace, $U^ \in$ Gr$(k^*,n)$ at the center of the minimum enclosing ball, that best represents the data.
Tasks
Published	2020-03-27
URL	https://arxiv.org/abs/2003.12455v1
PDF	https://arxiv.org/pdf/2003.12455v1.pdf
PWC	https://paperswithcode.com/paper/on-a-minimum-enclosing-ball-of-a-collection
Repo
Framework

Linear Regression without Correspondences via Concave Minimization


Title	Linear Regression without Correspondences via Concave Minimization
Authors	Liangzu Peng, Manolis C. Tsakiris
Abstract	Linear regression without correspondences concerns the recovery of a signal in the linear regression setting, where the correspondences between the observations and the linear functionals are unknown. The associated maximum likelihood function is NP-hard to compute when the signal has dimension larger than one. To optimize this objective function we reformulate it as a concave minimization problem, which we solve via branch-and-bound. This is supported by a computable search space to branch, an effective lower bounding scheme via convex envelope minimization and a refined upper bound, all naturally arising from the concave minimization reformulation. The resulting algorithm outperforms state-of-the-art methods for fully shuffled data and remains tractable for up to $8$-dimensional signals, an untouched regime in prior work.
Tasks
Published	2020-03-17
URL	https://arxiv.org/abs/2003.07706v1
PDF	https://arxiv.org/pdf/2003.07706v1.pdf
PWC	https://paperswithcode.com/paper/linear-regression-without-correspondences-via
Repo
Framework

AOL: Adaptive Online Learning for Human Trajectory Prediction in Dynamic Video Scenes


Title	AOL: Adaptive Online Learning for Human Trajectory Prediction in Dynamic Video Scenes
Authors	Manh Huynh, Gita Alaghband
Abstract	We present a novel adaptive online learning (AOL) framework to predict human movement trajectories in dynamic video scenes. Our framework learns and adapts to changes in the scene environment and generates best network weights for different scenarios. The framework can be applied to prediction models and improve their performance as it dynamically adjusts when it encounters changes in the scene and can apply the best training weights for predicting the next locations. We demonstrate this by integrating our framework with two existing prediction models: LSTM [3] and Future Person Location (FPL) [1]. Furthermore, we analyze the number of network weights for optimal performance and show that we can achieve real-time with a fixed number of networks using the least recently used (LRU) strategy for maintaining the most recently trained network weights. With extensive experiments, we show that our framework increases prediction accuracies of LSTM and FPL by ~17% and 28% on average, and up to ~50% for FPL on the worst case while achieving real-time (20fps).
Tasks	Trajectory Prediction
Published	2020-02-16
URL	https://arxiv.org/abs/2002.06666v1
PDF	https://arxiv.org/pdf/2002.06666v1.pdf
PWC	https://paperswithcode.com/paper/aol-adaptive-online-learning-for-human
Repo
Framework

perm2vec: Graph Permutation Selection for Decoding of Error Correction Codes using Self-Attention


Title	perm2vec: Graph Permutation Selection for Decoding of Error Correction Codes using Self-Attention
Authors	Nir Raviv, Avi Caciularu, Tomer Raviv, Jacob Goldberger, Yair Be’ery
Abstract	Error correction codes are integral part of communication applications, boosting the reliability of transmission. The optimal decoding of transmitted codewords is the maximum likelihood rule, which is NP-hard due to the curse of dimensionality. For practical realizations, suboptimal decoding algorithms are employed; yet limited theoretical insights prevents one from exploiting the full potential of these algorithms. One such insight is the choice of permutation in permutation decoding. We present a data-driven framework for permutation selection, combining domain knowledge with machine learning concepts such as node embedding and self-attention. Significant and consistent improvements in the bit error rate are introduced for all simulated codes, over the baseline decoders. To the best of the authors’ knowledge, this work is the first to leverage the benefits of the neural Transformer networks in physical layer communication systems.
Tasks
Published	2020-02-06
URL	https://arxiv.org/abs/2002.02315v1
PDF	https://arxiv.org/pdf/2002.02315v1.pdf
PWC	https://paperswithcode.com/paper/perm2vec-graph-permutation-selection-for
Repo
Framework

Longevity Associated Geometry Identified in Satellite Images: Sidewalks, Driveways and Hiking Trails


Title	Longevity Associated Geometry Identified in Satellite Images: Sidewalks, Driveways and Hiking Trails
Authors	Joshua J. Levy, Rebecca M. Lebeaux, Anne G. Hoen, Brock C. Christensen, Louis J. Vaickus, Todd A. MacKenzie
Abstract	Importance: Following a century of increase, life expectancy in the United States has stagnated and begun to decline in recent decades. Using satellite images and street view images prior work has demonstrated associations of the built environment with income, education, access to care and health factors such as obesity. However, assessment of learned image feature relationships with variation in crude mortality rate across the United States has been lacking. Objective: Investigate prediction of county-level mortality rates in the U.S. using satellite images. Design: Satellite images were extracted with the Google Static Maps application programming interface for 430 counties representing approximately 68.9% of the US population. A convolutional neural network was trained using crude mortality rates for each county in 2015 to predict mortality. Learned image features were interpreted using Shapley Additive Feature Explanations, clustered, and compared to mortality and its associated covariate predictors. Main Outcomes and Measures: County mortality was predicted using satellite images. Results: Predicted mortality from satellite images in a held-out test set of counties was strongly correlated to the true crude mortality rate (Pearson r=0.72). Learned image features were clustered, and we identified 10 clusters that were associated with education, income, geographical region, race and age. Conclusion and Relevance: The application of deep learning techniques to remotely-sensed features of the built environment can serve as a useful predictor of mortality in the United States. Tools that are able to identify image features associated with health-related outcomes can inform targeted public health interventions.
Tasks
Published	2020-03-05
URL	https://arxiv.org/abs/2003.08750v1
PDF	https://arxiv.org/pdf/2003.08750v1.pdf
PWC	https://paperswithcode.com/paper/longevity-associated-geometry-identified-in
Repo
Framework

Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation


Title	Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation
Authors	Javier Carnerero-Cano, Luis Muñoz-González, Phillippa Spencer, Emil C. Lupu
Abstract	Machine Learning (ML) algorithms are vulnerable to poisoning attacks, where a fraction of the training data can be manipulated to deliberately degrade the algorithms’ performance. Optimal poisoning attacks, which can be formulated as bilevel optimisation problems, help to assess the robustness of learning algorithms in worst-case scenarios. However, current attacks against algorithms with hyperparameters typically assume that these hyperparameters are constant and thus ignore the effect the attack has on them. In this paper, we show that this approach leads to an overly pessimistic view of the robustness of the learning algorithms tested. We propose a novel optimal attack formulation that considers the effect of the attack on the hyperparameters by modelling the attack as a multiobjective bilevel optimisation problem. We apply this novel attack formulation to ML classifiers using $L_2$ regularisation and show that, in contrast to results previously reported in the literature, $L_2$ regularisation enhances the stability of the learning algorithms and helps to partially mitigate poisoning attacks. Our empirical evaluation on different datasets confirms the limitations of previous poisoning attack strategies, evidences the benefits of using $L_2$ regularisation to dampen the effect of poisoning attacks and shows that the regularisation hyperparameter increases as more malicious data points are injected in the training dataset.
Tasks	bilevel optimization, data poisoning, L2 Regularization, Multiobjective Optimization
Published	2020-02-28
URL	https://arxiv.org/abs/2003.00040v1
PDF	https://arxiv.org/pdf/2003.00040v1.pdf
PWC	https://paperswithcode.com/paper/regularisation-can-mitigate-poisoning-attacks
Repo
Framework

Extracting more from boosted decision trees: A high energy physics case study


Title	Extracting more from boosted decision trees: A high energy physics case study
Authors	Vidhi Lalchand
Abstract	Particle identification is one of the core tasks in the data analysis pipeline at the Large Hadron Collider (LHC). Statistically, this entails the identification of rare signal events buried in immense backgrounds that mimic the properties of the former. In machine learning parlance, particle identification represents a classification problem characterized by overlapping and imbalanced classes. Boosted decision trees (BDTs) have had tremendous success in the particle identification domain but more recently have been overshadowed by deep learning (DNNs) approaches. This work proposes an algorithm to extract more out of standard boosted decision trees by targeting their main weakness, susceptibility to overfitting. This novel construction harnesses the meta-learning techniques of boosting and bagging simultaneously and performs remarkably well on the ATLAS Higgs (H) to tau-tau data set (ATLAS et al., 2014) which was the subject of the 2014 Higgs ML Challenge (Adam-Bourdarios et al., 2015). While the decay of Higgs to a pair of tau leptons was established in 2018 (CMS collaboration et al., 2017) at the 4.9$\sigma$ significance based on the 2016 data taking period, the 2014 public data set continues to serve as a benchmark data set to test the performance of supervised classification schemes. We show that the score achieved by the proposed algorithm is very close to the published winning score which leverages an ensemble of deep neural networks (DNNs). Although this paper focuses on a single application, it is expected that this simple and robust technique will find wider applications in high energy physics.
Tasks	Meta-Learning
Published	2020-01-16
URL	https://arxiv.org/abs/2001.06033v1
PDF	https://arxiv.org/pdf/2001.06033v1.pdf
PWC	https://paperswithcode.com/paper/extracting-more-from-boosted-decision-trees-a
Repo
Framework

SQUIRL: Robust and Efficient Learning from Video Demonstration of Long-Horizon Robotic Manipulation Tasks


Title	SQUIRL: Robust and Efficient Learning from Video Demonstration of Long-Horizon Robotic Manipulation Tasks
Authors	Bohan Wu, Feng Xu, Zhanpeng He, Abhi Gupta, Peter K. Allen
Abstract	Recent advances in deep reinforcement learning (RL) have demonstrated its potential to learn complex robotic manipulation tasks. However, RL still requires the robot to collect a large amount of real-world experience. To address this problem, recent works have proposed learning from expert demonstrations (LfD), particularly via inverse reinforcement learning (IRL), given its ability to achieve robust performance with only a small number of expert demonstrations. Nevertheless, deploying IRL on real robots is still challenging due to the large number of robot experiences it requires. This paper aims to address this scalability challenge with a robust, sample-efficient, and general meta-IRL algorithm, SQUIRL, that performs a new but related long-horizon task robustly given only a single video demonstration. First, this algorithm bootstraps the learning of a task encoder and a task-conditioned policy using behavioral cloning (BC). It then collects real-robot experiences and bypasses reward learning by directly recovering a Q-function from the combined robot and expert trajectories. Next, this algorithm uses the Q-function to re-evaluate all cumulative experiences collected by the robot to improve the policy quickly. In the end, the policy performs more robustly (90%+ success) than BC on new tasks while requiring no trial-and-errors at test time. Finally, our real-robot and simulated experiments demonstrate our algorithm’s generality across different state spaces, action spaces, and vision-based manipulation tasks, e.g., pick-pour-place and pick-carry-drop.
Tasks
Published	2020-03-10
URL	https://arxiv.org/abs/2003.04956v1
PDF	https://arxiv.org/pdf/2003.04956v1.pdf
PWC	https://paperswithcode.com/paper/squirl-robust-and-efficient-learning-from
Repo
Framework

ETRI-Activity3D: A Large-Scale RGB-D Dataset for Robots to Recognize Daily Activities of the Elderly


Title	ETRI-Activity3D: A Large-Scale RGB-D Dataset for Robots to Recognize Daily Activities of the Elderly
Authors	Jinhyeok Jang, Dohyung Kim, Cheonshu Park, Minsu Jang, Jaeyeon Lee, Jaehong Kim
Abstract	Deep learning, based on which many modern algorithms operate, is well known to be data-hungry. In particular, the datasets appropriate for the intended application are difficult to obtain. To cope with this situation, we introduce a new dataset called ETRI-Activity3D, focusing on the daily activities of the elderly in robot-view. The major characteristics of the new dataset are as follows: 1) practical action categories that are selected from the close observation of the daily lives of the elderly; 2) realistic data collection, which reflects the robot’s working environment and service situations; and 3) a large-scale dataset that overcomes the limitations of the current 3D activity analysis benchmark datasets. The proposed dataset contains 112,620 samples including RGB videos, depth maps, and skeleton sequences. During the data acquisition, 100 subjects were asked to perform 55 daily activities. Additionally, we propose a novel network called four-stream adaptive CNN (FSA-CNN). The proposed FSA-CNN has three main properties: robustness to spatio-temporal variations, input-adaptive activation function, and extension of the conventional two-stream approach. In the experiment section, we confirmed the superiority of the proposed FSA-CNN using NTU RGB+D and ETRI-Activity3D. Further, the domain difference between both groups of age was verified experimentally. Finally, the extension of FSA-CNN to deal with the multimodal data was investigated.
Tasks
Published	2020-03-04
URL	https://arxiv.org/abs/2003.01920v2
PDF	https://arxiv.org/pdf/2003.01920v2.pdf
PWC	https://paperswithcode.com/paper/etri-activity3d-a-large-scale-rgb-d-dataset
Repo
Framework

Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics


Title	Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics
Authors	Michael Neunert, Abbas Abdolmaleki, Markus Wulfmeier, Thomas Lampe, Jost Tobias Springenberg, Roland Hafner, Francesco Romano, Jonas Buchli, Nicolas Heess, Martin Riedmiller
Abstract	Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, control gains or analogue outputs. However, when defining the corresponding optimal control or reinforcement learning problem, it is commonly approximated with fully continuous or fully discrete action spaces. These simplifications aim at tailoring the problem to a particular algorithm or solver which may only support one type of action space. Alternatively, expert heuristics are used to remove discrete actions from an otherwise continuous space. In contrast, we propose to treat hybrid problems in their ‘native’ form by solving them with hybrid reinforcement learning, which optimizes for discrete and continuous actions simultaneously. In our experiments, we first demonstrate that the proposed approach efficiently solves such natively hybrid reinforcement learning problems. We then show, both in simulation and on robotic hardware, the benefits of removing possibly imperfect expert-designed heuristics. Lastly, hybrid reinforcement learning encourages us to rethink problem definitions. We propose reformulating control problems, e.g. by adding meta actions, to improve exploration or reduce mechanical wear and tear.
Tasks
Published	2020-01-02
URL	https://arxiv.org/abs/2001.00449v1
PDF	https://arxiv.org/pdf/2001.00449v1.pdf
PWC	https://paperswithcode.com/paper/continuous-discrete-reinforcement-learning
Repo
Framework

Privacy-Preserving Image Classification in the Local Setting


Title	Privacy-Preserving Image Classification in the Local Setting
Authors	Sen Wang, J. Morris Chang
Abstract	Image data has been greatly produced by individuals and commercial vendors in the daily life, and it has been used across various domains, like advertising, medical and traffic analysis. Recently, image data also appears to be greatly important in social utility, like emergency response. However, the privacy concern becomes the biggest obstacle that prevents further exploration of image data, due to that the image could reveal sensitive information, like the personal identity and locations. The recent developed Local Differential Privacy (LDP) brings us a promising solution, which allows the data owners to randomly perturb their input to provide the plausible deniability of the data before releasing. In this paper, we consider a two-party image classification problem, in which data owners hold the image and the untrustworthy data user would like to fit a machine learning model with these images as input. To protect the image privacy, we propose to locally perturb the image representation before revealing to the data user. Subsequently, we analyze how the perturbation satisfies {\epsilon}-LDP and affect the data utility regarding count-based and distance-based machine learning algorithm, and propose a supervised image feature extractor, DCAConv, which produces an image representation with scalable domain size. Our experiments show that DCAConv could maintain a high data utility while preserving the privacy regarding multiple image benchmark datasets.
Tasks	Image Classification
Published	2020-02-09
URL	https://arxiv.org/abs/2002.03261v1
PDF	https://arxiv.org/pdf/2002.03261v1.pdf
PWC	https://paperswithcode.com/paper/privacy-preserving-image-classification-in
Repo
Framework

FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection


Title	FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection
Authors	Ruixuan Liu, Yang Cao, Masatoshi Yoshikawa, Hong Chen
Abstract	As massive data are produced from small gadgets, federated learning on mobile devices has become an emerging trend. In the federated setting, Stochastic Gradient Descent (SGD) has been widely used in federated learning for various machine learning models. To prevent privacy leakages from gradients that are calculated on users’ sensitive data, local differential privacy (LDP) has been considered as a privacy guarantee in federated SGD recently. However, the existing solutions have a dimension dependency problem: the injected noise is substantially proportional to the dimension $d$. In this work, we propose a two-stage framework FedSel for federated SGD under LDP to relieve this problem. Our key idea is that not all dimensions are equally important so that we privately select Top-k dimensions according to their contributions in each iteration of federated SGD. Specifically, we propose three private dimension selection mechanisms and adapt the gradient accumulation technique to stabilize the learning process with noisy updates. We also theoretically analyze privacy, accuracy and time complexity of FedSel, which outperforms the state-of-the-art solutions. Experiments on real-world and synthetic datasets verify the effectiveness and efficiency of our framework.
Tasks
Published	2020-03-24
URL	https://arxiv.org/abs/2003.10637v1
PDF	https://arxiv.org/pdf/2003.10637v1.pdf
PWC	https://paperswithcode.com/paper/fedsel-federated-sgd-under-local-differential
Repo
Framework