April 1, 2020

2964 words 14 mins read

Paper Group ANR 396

Privacy Amplification of Iterative Algorithms via Contraction Coefficients. hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition. Towards Automated Statistical Physics : Data-driven Modeling of Complex Systems with Deep Learning. Don’t Judge an Object by Its Context: Learning to Overcome Contextual Bias. Demonstrating R …

Privacy Amplification of Iterative Algorithms via Contraction Coefficients


Title	Privacy Amplification of Iterative Algorithms via Contraction Coefficients
Authors	Shahab Asoodeh, Mario Diaz, Flavio P. Calmon
Abstract	We investigate the framework of privacy amplification by iteration, recently proposed by Feldman et al., from an information-theoretic lens. We demonstrate that differential privacy guarantees of iterative mappings can be determined by a direct application of contraction coefficients derived from strong data processing inequalities for $f$-divergences. In particular, by generalizing the Dobrushin’s contraction coefficient for total variation distance to an $f$-divergence known as $E_{\gamma}$-divergence, we derive tighter bounds on the differential privacy parameters of the projected noisy stochastic gradient descent algorithm with hidden intermediate updates.
Tasks
Published	2020-01-17
URL	https://arxiv.org/abs/2001.06546v1
PDF	https://arxiv.org/pdf/2001.06546v1.pdf
PWC	https://paperswithcode.com/paper/privacy-amplification-of-iterative-algorithms
Repo
Framework

hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition


Title	hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
Authors	Ehsan Kharazmi, Zhongqiang Zhang, George Em Karniadakis
Abstract	We formulate a general framework for hp-variational physics-informed neural networks (hp-VPINNs) based on the nonlinear approximation of shallow and deep neural networks and hp-refinement via domain decomposition and projection onto space of high-order polynomials. The trial space is the space of neural network, which is defined globally over the whole computational domain, while the test space contains the piecewise polynomials. Specifically in this study, the hp-refinement corresponds to a global approximation with local learning algorithm that can efficiently localize the network parameter optimization. We demonstrate the advantages of hp-VPINNs in accuracy and training cost for several numerical examples of function approximation and solving differential equations.
Tasks
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05385v1
PDF	https://arxiv.org/pdf/2003.05385v1.pdf
PWC	https://paperswithcode.com/paper/hp-vpinns-variational-physics-informed-neural
Repo
Framework

Towards Automated Statistical Physics : Data-driven Modeling of Complex Systems with Deep Learning


Title	Towards Automated Statistical Physics : Data-driven Modeling of Complex Systems with Deep Learning
Authors	Seungwoong Ha, Hawoong Jeong
Abstract	Rich phenomena from complex systems have long intrigued researchers, and yet modeling system micro-dynamics and inferring the forms of interaction are challenging for conventional data-driven approaches, being generally established by human scientists. In this study, we propose AgentNet, a generalized data-driven framework to analyze and understand the hidden interactions in complex systems. AgentNet utilizes a graph attention network to model the interaction between individual agents and employs various encoders and decoders that can be selectively applied to any desired system. Our model successfully captured three different simulated complex systems, namely cellular automata, the Vicsek model, and active Ornstein-Uhlenbeck particles in which, notably, AgentNet’s visualized attention values coincided with the true interaction strength. Demonstration with empirical data from a flock of birds showed that AgentNet prediction could yield the qualitatively same collective phenomena as exhibited by real birds. We expect our framework to open a novel path to investigating complex systems and to provide insight into process-driven modeling.
Tasks
Published	2020-01-03
URL	https://arxiv.org/abs/2001.02539v2
PDF	https://arxiv.org/pdf/2001.02539v2.pdf
PWC	https://paperswithcode.com/paper/towards-automated-statistical-physics-data
Repo
Framework

Don’t Judge an Object by Its Context: Learning to Overcome Contextual Bias


Title	Don’t Judge an Object by Its Context: Learning to Overcome Contextual Bias
Authors	Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman, Yong Jae Lee, Matt Feiszli, Deepti Ghadiyaram
Abstract	Existing models often leverage co-occurrences between objects and their context to improve recognition accuracy. However, strongly relying on context risks a model’s generalizability, especially when typical co-occurrence patterns are absent. This work focuses on addressing such contextual biases to improve the robustness of the learnt feature representations. Our goal is to accurately recognize a category in the absence of its context, without compromising on performance when it co-occurs with context. Our key idea is to decorrelate feature representations of a category from its co-occurring context. We achieve this by learning a feature subspace that explicitly represents categories occurring in the absence of context along side a joint feature subspace that represents both categories and context. Our very simple yet effective method is extensible to two multi-label tasks – object and attribute classification. On 4 challenging datasets, we demonstrate the effectiveness of our method in reducing contextual bias.
Tasks
Published	2020-01-09
URL	https://arxiv.org/abs/2001.03152v1
PDF	https://arxiv.org/pdf/2001.03152v1.pdf
PWC	https://paperswithcode.com/paper/dont-judge-an-object-by-its-context-learning
Repo
Framework

Demonstrating Rosa: the fairness solution for any Data Analytic pipeline


Title	Demonstrating Rosa: the fairness solution for any Data Analytic pipeline
Authors	Kate Wilkinson, George Cevora
Abstract	Most datasets of interest to the analytics industry are impacted by various forms of human bias. The outcomes of Data Analytics [DA] or Machine Learning [ML] on such data are therefore prone to replicating the bias. As a result, a large number of biased decision-making systems based on DA/ML have recently attracted attention. In this paper we introduce Rosa, a free, web-based tool to easily de-bias datasets with respect to a chosen characteristic. Rosa is based on the principles of Fair Adversarial Networks, developed by illumr Ltd., and can therefore remove interactive, non-linear, and non-binary bias. Rosa is stand-alone pre-processing step / API, meaning it can be used easily with any DA/ML pipeline. We test the efficacy of Rosa in removing bias from data-driven decision making systems by performing standard DA tasks on five real-world datasets, selected for their relevance to current DA problems, and also their high potential for bias. We use simple ML models to model a characteristic of analytical interest, and compare the level of bias in the model output both with and without Rosa as a pre-processing step. We find that in all cases there is a substantial decrease in bias of the data-driven decision making systems when the data is pre-processed with Rosa.
Tasks	Decision Making
Published	2020-02-28
URL	https://arxiv.org/abs/2003.00899v1
PDF	https://arxiv.org/pdf/2003.00899v1.pdf
PWC	https://paperswithcode.com/paper/demonstrating-rosa-the-fairness-solution-for
Repo
Framework

Guess First to Enable Better Compression and Adversarial Robustness


Title	Guess First to Enable Better Compression and Adversarial Robustness
Authors	Sicheng Zhu, Bang An, Shiyu Niu
Abstract	Machine learning models are generally vulnerable to adversarial examples, which is in contrast to the robustness of humans. In this paper, we try to leverage one of the mechanisms in human recognition and propose a bio-inspired classification framework in which model inference is conditioned on label hypothesis. We provide a class of training objectives for this framework and an information bottleneck regularizer which utilizes the advantage that label information can be discarded during inference. This framework enables better compression of the mutual information between inputs and latent representations without loss of learning capacity, at the cost of tractable inference complexity. Better compression and elimination of label information further bring better adversarial robustness without loss of natural accuracy, which is demonstrated in the experiment.
Tasks
Published	2020-01-10
URL	https://arxiv.org/abs/2001.03311v1
PDF	https://arxiv.org/pdf/2001.03311v1.pdf
PWC	https://paperswithcode.com/paper/guess-first-to-enable-better-compression-and
Repo
Framework

SELD-TCN: Sound Event Localization & Detection via Temporal Convolutional Networks


Title	SELD-TCN: Sound Event Localization & Detection via Temporal Convolutional Networks
Authors	Karim Guirguis, Christoph Schorn, Andre Guntoro, Sherif Abdulatif, Bin Yang
Abstract	The understanding of the surrounding environment plays a critical role in autonomous robotic systems, such as self-driving cars. Extensive research has been carried out concerning visual perception. Yet, to obtain a more complete perception of the environment, autonomous systems of the future should also take acoustic information into account. Recent sound event localization and detection (SELD) frameworks utilize convolutional recurrent neural networks (CRNNs). However, considering the recurrent nature of CRNNs, it becomes challenging to implement them efficiently on embedded hardware. Not only are their computations strenuous to parallelize, but they also require high memory bandwidth and large memory buffers. In this work, we develop a more robust and hardware-friendly novel architecture based on a temporal convolutional network(TCN). The proposed framework (SELD-TCN) outperforms the state-of-the-art SELDnet performance on four different datasets. Moreover, SELD-TCN achieves 4x faster training time per epoch and 40x faster inference time on an ordinary graphics processing unit (GPU).
Tasks	Self-Driving Cars
Published	2020-03-03
URL	https://arxiv.org/abs/2003.01609v1
PDF	https://arxiv.org/pdf/2003.01609v1.pdf
PWC	https://paperswithcode.com/paper/seld-tcn-sound-event-localization-detection
Repo
Framework

Self-Supervised 2D Image to 3D Shape Translation with Disentangled Representations


Title	Self-Supervised 2D Image to 3D Shape Translation with Disentangled Representations
Authors	Berk Kaya, Radu Timofte
Abstract	We present a framework to translate between 2D image views and 3D object shapes. Recent progress in deep learning enabled us to learn structure-aware representations from a scene. However, the existing literature assumes that pairs of images and 3D shapes are available for training in full supervision. In this paper, we propose SIST, a Self-supervised Image to Shape Translation framework that fulfills three tasks: (i) reconstructing the 3D shape from a single image; (ii) learning disentangled representations for shape, appearance and viewpoint; and (iii) generating a realistic RGB image from these independent factors. In contrast to the existing approaches, our method does not require image-shape pairs for training. Instead, it uses unpaired image and shape datasets from the same object class and jointly trains image generator and shape reconstruction networks. Our translation method achieves promising results, comparable in quantitative and qualitative terms to the state-of-the-art achieved by fully-supervised methods.
Tasks
Published	2020-03-22
URL	https://arxiv.org/abs/2003.10016v1
PDF	https://arxiv.org/pdf/2003.10016v1.pdf
PWC	https://paperswithcode.com/paper/self-supervised-2d-image-to-3d-shape
Repo
Framework

Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar


Title	Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar
Authors	Dan Barnes, Ingmar Posner
Abstract	This paper presents a self-supervised framework for learning to detect robust keypoints for odometry estimation and metric localisation in radar. By embedding a differentiable point-based motion estimator inside our architecture, we learn keypoint locations, scores and descriptors from localisation error alone. This approach avoids imposing any assumption on what makes a robust keypoint and crucially allows them to be optimised for our application. Furthermore the architecture is sensor agnostic and can be applied to most modalities. We run experiments on 280km of real world driving from the Oxford Radar RobotCar Dataset and improve on the state-of-the-art in point-based radar odometry, reducing errors by up to 45% whilst running an order of magnitude faster, simultaneously solving metric loop closures. Combining these outputs, we provide a framework capable of full mapping and localisation with radar in urban environments.
Tasks
Published	2020-01-29
URL	https://arxiv.org/abs/2001.10789v3
PDF	https://arxiv.org/pdf/2001.10789v3.pdf
PWC	https://paperswithcode.com/paper/under-the-radar-learning-to-predict-robust
Repo
Framework

Fairness by Explicability and Adversarial SHAP Learning


Title	Fairness by Explicability and Adversarial SHAP Learning
Authors	James M. Hickey, Pietro G. Di Stefano, Vlasios Vasileiou
Abstract	The ability to understand and trust the fairness of model predictions, particularly when considering the outcomes of unprivileged groups, is critical to the deployment and adoption of machine learning systems. SHAP values provide a unified framework for interpreting model predictions and feature attribution but do not address the problem of fairness directly. In this work, we propose a new definition of fairness that emphasises the role of an external auditor and model explicability. To satisfy this definition, we develop a framework for mitigating model bias using regularizations constructed from the SHAP values of an adversarial surrogate model. We focus on the binary classification task with a single unprivileged group and link our fairness explicability constraints to classical statistical fairness metrics. We demonstrate our approaches using gradient and adaptive boosting on: a synthetic dataset, the UCI Adult (Census) dataset and a real-world credit scoring dataset. The models produced were fairer and performant.
Tasks
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05330v2
PDF	https://arxiv.org/pdf/2003.05330v2.pdf
PWC	https://paperswithcode.com/paper/fairness-by-explicability-and-adversarial
Repo
Framework

Extrapolation Towards Imaginary $0$-Nearest Neighbour and Its Improved Convergence Rate


Title	Extrapolation Towards Imaginary $0$-Nearest Neighbour and Its Improved Convergence Rate
Authors	Akifumi Okuno, Hidetoshi Shimodaira
Abstract	$k$-nearest neighbour ($k$-NN) is one of the simplest and most widely-used methods for supervised classification, that predicts a query’s label by taking weighted ratio of observed labels of $k$ objects nearest to the query. The weights and the parameter $k \in \mathbb{N}$ regulate its bias-variance trade-off, and the trade-off implicitly affects the convergence rate of the excess risk for the $k$-NN classifier; several existing studies considered selecting optimal $k$ and weights to obtain faster convergence rate. Whereas $k$-NN with non-negative weights has been developed widely, it was proved that negative weights are essential for eradicating the bias terms and attaining optimal convergence rate. However, computation of the optimal weights requires solving entangled equations. Thus, other simpler approaches that can find optimal real-valued weights are appreciated in practice. In this paper, we propose multiscale $k$-NN (MS-$k$-NN), that extrapolates unweighted $k$-NN estimators from several $k \ge 1$ values to $k=0$, thus giving an imaginary 0-NN estimator. MS-$k$-NN implicitly corresponds to an adaptive method for finding favorable real-valued weights, and we theoretically prove that the MS-$k$-NN attains the improved rate, that coincides with the existing optimal rate under some conditions.
Tasks
Published	2020-02-08
URL	https://arxiv.org/abs/2002.03054v1
PDF	https://arxiv.org/pdf/2002.03054v1.pdf
PWC	https://paperswithcode.com/paper/extrapolation-towards-imaginary-0-nearest
Repo
Framework

Fine-grained Uncertainty Modeling in Neural Networks


Title	Fine-grained Uncertainty Modeling in Neural Networks
Authors	Rahul Soni, Naresh Shah, Jimmy D. Moore
Abstract	Existing uncertainty modeling approaches try to detect an out-of-distribution point from the in-distribution dataset. We extend this argument to detect finer-grained uncertainty that distinguishes between (a). certain points, (b). uncertain points but within the data distribution, and (c). out-of-distribution points. Our method corrects overconfident NN decisions, detects outlier points and learns to say ``I don’t know’’ when uncertain about a critical point between the top two predictions. In addition, we provide a mechanism to quantify class distributions overlap in the decision manifold and investigate its implications in model interpretability. Our method is two-step: in the first step, the proposed method builds a class distribution using Kernel Activation Vectors (kav) extracted from the Network. In the second step, the algorithm determines the confidence of a test point by a hierarchical decision rule based on the chi-squared distribution of squared Mahalanobis distances. Our method sits on top of a given Neural Network, requires a single scan of training data to estimate class distribution statistics, and is highly scalable to deep networks and wider pre-softmax layer. As a positive side effect, our method helps to prevent adversarial attacks without requiring any additional training. It is directly achieved when the Softmax layer is substituted by our robust uncertainty layer at the evaluation phase. \|
Tasks
Published	2020-02-11
URL	https://arxiv.org/abs/2002.04205v1
PDF	https://arxiv.org/pdf/2002.04205v1.pdf
PWC	https://paperswithcode.com/paper/fine-grained-uncertainty-modeling-in-neural
Repo
Framework

Geometric Proxies for Live RGB-D Stream Enhancement and Consolidation


Title	Geometric Proxies for Live RGB-D Stream Enhancement and Consolidation
Authors	Adrien Kaiser, José Alonso Ybanez Zepeda, Tamy Boubekeur
Abstract	We propose a geometric superstructure for unified real-time processing of RGB-D data. Modern RGB-D sensors are widely used for indoor 3D capture, with applications ranging from modeling to robotics, through augmented reality. Nevertheless, their use is limited by their low resolution, with frames often corrupted with noise, missing data and temporal inconsistencies. Our approach consists in generating and updating through time a single set of compact local statistics parameterized over detected geometric proxies, which are fed from raw RGB-D data. Our proxies provide several processing primitives, which improve the quality of the RGB-D stream on the fly or lighten further operations. Experimental results confirm that our lightweight analysis framework copes well with embedded execution as well as moderate memory and computational capabilities compared to state-of-the-art methods. Processing RGB-D data with our proxies allows noise and temporal flickering removal, hole filling and resampling. As a substitute of the observed scene, our proxies can additionally be applied to compression and scene reconstruction. We present experiments performed with our framework in indoor scenes of different natures within a recent open RGB-D dataset.
Tasks
Published	2020-01-21
URL	https://arxiv.org/abs/2001.07577v1
PDF	https://arxiv.org/pdf/2001.07577v1.pdf
PWC	https://paperswithcode.com/paper/geometric-proxies-for-live-rgb-d-stream
Repo
Framework

Localized sketching for matrix multiplication and ridge regression


Title	Localized sketching for matrix multiplication and ridge regression
Authors	Rakshith S Srinivasa, Mark A Davenport, Justin Romberg
Abstract	We consider sketched approximate matrix multiplication and ridge regression in the novel setting of localized sketching, where at any given point, only part of the data matrix is available. This corresponds to a block diagonal structure on the sketching matrix. We show that, under mild conditions, block diagonal sketching matrices require only O(stable rank / \epsilon^2) and $O( stat. dim. \epsilon)$ total sample complexity for matrix multiplication and ridge regression, respectively. This matches the state-of-the-art bounds that are obtained using global sketching matrices. The localized nature of sketching considered allows for different parts of the data matrix to be sketched independently and hence is more amenable to computation in distributed and streaming settings and results in a smaller memory and computational footprint.
Tasks
Published	2020-03-20
URL	https://arxiv.org/abs/2003.09097v1
PDF	https://arxiv.org/pdf/2003.09097v1.pdf
PWC	https://paperswithcode.com/paper/localized-sketching-for-matrix-multiplication
Repo
Framework

Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning


Title	Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning
Authors	Navid Naderializadeh, Jaroslaw Sydir, Meryem Simsek, Hosein Nikopour
Abstract	We propose a mechanism for distributed radio resource management using multi-agent deep reinforcement learning (RL) for interference mitigation in wireless networks. We equip each transmitter in the network with a deep RL agent, which receives partial delayed observations from its associated users, while also exchanging observations with its neighboring agents, and decides on which user to serve and what transmit power to use at each scheduling interval. Our proposed framework enables the agents to make decisions simultaneously and in a distributed manner, without any knowledge about the concurrent decisions of other agents. Moreover, our design of the agents’ observation and action spaces is scalable, in the sense that an agent trained on a scenario with a specific number of transmitters and receivers can be readily applied to scenarios with different numbers of transmitters and/or receivers. Simulation results demonstrate the superiority of our proposed approach compared to decentralized baselines in terms of the tradeoff between average and $5^{th}$ percentile user rates, while achieving performance close to, and even in certain cases outperforming, that of a centralized information-theoretic scheduling algorithm. We also show that our trained agents are robust and maintain their performance gains when experiencing mismatches between training and testing deployments.
Tasks
Published	2020-02-14
URL	https://arxiv.org/abs/2002.06215v1
PDF	https://arxiv.org/pdf/2002.06215v1.pdf
PWC	https://paperswithcode.com/paper/resource-management-in-wireless-networks-via
Repo
Framework