Paper Group NANR 20
Training individually fair ML models with sensitive subspace robustness. Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation. Effort-value payoff in lemmatisation for Uralic languages. Proceedings of the Sixth International Workshop on Computational Linguistics of Uralic Languages. Removing the Representation Error of GAN …
Training individually fair ML models with sensitive subspace robustness
| Title | Training individually fair ML models with sensitive subspace robustness |
| Authors | Anonymous |
| Abstract | We propose an approach to training machine learning models that are fair in the sense that their performance is invariant under certain perturbations to the features. For example, the performance of a resume screening system should be invariant under changes to the name of the applicant. We formalize this intuitive notion of fairness by connecting it to the original notion of individual fairness put forth by Dwork et al and show that the proposed approach achieves this notion of fairness. We also demonstrate the effectiveness of the approach on two machine learning tasks that are susceptible to gender and racial biases. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=B1gdkxHFDH |
| https://openreview.net/pdf?id=B1gdkxHFDH | |
| PWC | https://paperswithcode.com/paper/training-individually-fair-ml-models-with |
| Repo | |
| Framework | |
Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation
| Title | Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation |
| Authors | Anonymous |
| Abstract | Convolutional networks are not aware of an object’s geometric variations, which leads to inefficient utilization of model and data capacity. To overcome this issue, recent works on deformation modeling seek to spatially reconfigure the data towards a common arrangement such that semantic recognition suffers less from deformation. This is typically done by augmenting static operators with learned free-form sampling grids in the image space, dynamically tuned to the data and task for adapting the receptive field. Yet adapting the receptive field does not quite reach the actual goal – what really matters to the network is the effective receptive field (ERF), which reflects how much each pixel contributes. It is thus natural to design other approaches to adapt the ERF directly during runtime. In this work, we instantiate one possible solution as Deformable Kernels (DKs), a family of novel and generic convolutional operators for handling object deformations by directly adapting the ERF while leaving the receptive field untouched. At the heart of our method is the ability to resample the original kernel space towards recovering the deformation of objects. This approach is justified with theoretical insights that the ERF is strictly determined by data sampling locations and kernel values. We implement DKs as generic drop-in replacements of rigid kernels and conduct a series of empirical studies whose results conform with our theories. Over several tasks and standard base models, our approach compares favorably against prior works that adapt during runtime. In addition, further experiments suggest a working mechanism orthogonal and complementary to previous works. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SkxSv6VFvS |
| https://openreview.net/pdf?id=SkxSv6VFvS | |
| PWC | https://paperswithcode.com/paper/deformable-kernels-adapting-effective-1 |
| Repo | |
| Framework | |
Effort-value payoff in lemmatisation for Uralic languages
| Title | Effort-value payoff in lemmatisation for Uralic languages |
| Authors | Nick Howell, Maria Bibaeva, Francis M. Tyers |
| Abstract | |
| Tasks | |
| Published | 2020-10-01 |
| URL | https://www.aclweb.org/anthology/2020.iwclul-1.2/ |
| https://www.aclweb.org/anthology/2020.iwclul-1.2 | |
| PWC | https://paperswithcode.com/paper/effort-value-payoff-in-lemmatisation-for |
| Repo | |
| Framework | |
Proceedings of the Sixth International Workshop on Computational Linguistics of Uralic Languages
| Title | Proceedings of the Sixth International Workshop on Computational Linguistics of Uralic Languages |
| Authors | |
| Abstract | |
| Tasks | |
| Published | 2020-10-01 |
| URL | https://www.aclweb.org/anthology/2020.iwclul-1.0/ |
| https://www.aclweb.org/anthology/2020.iwclul-1.0 | |
| PWC | https://paperswithcode.com/paper/proceedings-of-the-sixth-international-2 |
| Repo | |
| Framework | |
Removing the Representation Error of GAN Image Priors Using the Deep Decoder
| Title | Removing the Representation Error of GAN Image Priors Using the Deep Decoder |
| Authors | Anonymous |
| Abstract | Generative models, such as GANs, have demonstrated impressive performance as natural image priors for solving inverse problems such as image restoration and compressive sensing. Despite this performance, they can exhibit substantial representation error for both in-distribution and out-of-distribution images, because they maintain explicit low-dimensional learned representations of a natural signal class. In this paper, we demonstrate a method for removing the representation error of a GAN when used as a prior in inverse problems by modeling images as the linear combination of a GAN with a Deep Decoder. The deep decoder is an underparameterized and most importantly unlearned natural signal model similar to the Deep Image Prior. No knowledge of the specific inverse problem is needed in the training of the GAN underlying our method. For compressive sensing and image superresolution, our hybrid model exhibits consistently higher PSNRs than both the GAN priors and Deep Decoder separately, both on in-distribution and out-of-distribution images. This model provides a method for extensibly and cheaply leveraging both the benefits of learned and unlearned image recovery priors in inverse problems. |
| Tasks | Compressive Sensing, Image Restoration |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=rkegcC4YvS |
| https://openreview.net/pdf?id=rkegcC4YvS | |
| PWC | https://paperswithcode.com/paper/removing-the-representation-error-of-gan |
| Repo | |
| Framework | |
Differentiable Hebbian Consolidation for Continual Learning
| Title | Differentiable Hebbian Consolidation for Continual Learning |
| Authors | Anonymous |
| Abstract | Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge. However, catastrophic forgetting poses a grand challenge for neural networks performing such learning process. Thus, neural networks that are deployed in the real world often struggle in scenarios where the data distribution is non-stationary (concept drift), imbalanced, or not always fully available, i.e., rare edge cases. We propose a Differentiable Hebbian Consolidation model which is composed of a Differentiable Hebbian Plasticity (DHP) Softmax layer that adds a rapid learning plastic component (compressed episodic memory) to the fixed (slow changing) parameters of the softmax output layer; enabling learned representations to be retained for a longer timescale. We demonstrate the flexibility of our method by integrating well-known task-specific synaptic consolidation methods to penalize changes in the slow weights that are important for each target task. We evaluate our approach on the Permuted MNIST, Split MNIST and Vision Datasets Mixture benchmarks, and introduce an imbalanced variant of Permuted MNIST — a dataset that combines the challenges of class imbalance and concept drift. Our proposed model requires no additional hyperparameters and outperforms comparable baselines by reducing forgetting. |
| Tasks | Continual Learning |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BJlA6eBtvH |
| https://openreview.net/pdf?id=BJlA6eBtvH | |
| PWC | https://paperswithcode.com/paper/differentiable-hebbian-consolidation-for |
| Repo | |
| Framework | |
Skew-Explore: Learn faster in continuous spaces with sparse rewards
| Title | Skew-Explore: Learn faster in continuous spaces with sparse rewards |
| Authors | Anonymous |
| Abstract | In many reinforcement learning settings, rewards which are extrinsically available to the learning agent are too sparse to train a suitable policy. Beside reward shaping which requires human expertise, utilizing better exploration strategies helps to circumvent the problem of policy training with sparse rewards. In this work, we introduce an exploration approach based on maximizing the entropy of the visited states while learning a goal-conditioned policy. The main contribution of this work is to introduce a novel reward function which combined with a goal proposing scheme, increases the entropy of the visited states faster compared to the prior work. This improves the exploration capability of the agent, and therefore enhances the agent’s chance to solve sparse reward problems more efficiently. Our empirical studies demonstrate the superiority of the proposed method to solve different sparse reward problems in comparison to the prior work. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HJewxlHFwH |
| https://openreview.net/pdf?id=HJewxlHFwH | |
| PWC | https://paperswithcode.com/paper/skew-explore-learn-faster-in-continuous |
| Repo | |
| Framework | |
On Editing Dictionaries for Uralic Languages in an Online Environment
| Title | On Editing Dictionaries for Uralic Languages in an Online Environment |
| Authors | Khalid Alnajjar, Mika H{"a}m{"a}l{"a}inen, Jack Rueter |
| Abstract | |
| Tasks | |
| Published | 2020-10-01 |
| URL | https://www.aclweb.org/anthology/2020.iwclul-1.4/ |
| https://www.aclweb.org/anthology/2020.iwclul-1.4 | |
| PWC | https://paperswithcode.com/paper/on-editing-dictionaries-for-uralic-languages |
| Repo | |
| Framework | |
Hunting for antiharmonic stems in Erzya
| Title | Hunting for antiharmonic stems in Erzya |
| Authors | L{'a}szl{'o} Fejes |
| Abstract | |
| Tasks | |
| Published | 2020-10-01 |
| URL | https://www.aclweb.org/anthology/2020.iwclul-1.6/ |
| https://www.aclweb.org/anthology/2020.iwclul-1.6 | |
| PWC | https://paperswithcode.com/paper/hunting-for-antiharmonic-stems-in-erzya |
| Repo | |
| Framework | |
On the questions in developing computational infrastructure for Komi-Permyak
| Title | On the questions in developing computational infrastructure for Komi-Permyak |
| Authors | Jack Rueter, Niko Partanen, Larisa Ponomareva |
| Abstract | |
| Tasks | |
| Published | 2020-10-01 |
| URL | https://www.aclweb.org/anthology/2020.iwclul-1.3/ |
| https://www.aclweb.org/anthology/2020.iwclul-1.3 | |
| PWC | https://paperswithcode.com/paper/on-the-questions-in-developing-computational |
| Repo | |
| Framework | |
Meta-Learning Deep Energy-Based Memory Models
| Title | Meta-Learning Deep Energy-Based Memory Models |
| Authors | Anonymous |
| Abstract | We study the problem of learning associative memory – a system which is able to retrieve a remembered pattern based on its distorted or incomplete version. Attractor networks provide a sound model of associative memory: patterns are stored as attractors of the network dynamics and associative retrieval is performed by running the dynamics starting from a query pattern until it converges to an attractor. In such models the dynamics are often implemented as an optimization procedure that minimizes an energy function, such as in the classical Hopfield network. In general it is difficult to derive a writing rule for a given dynamics and energy that is both compressive and fast. Thus, most research in energy-based memory has been limited either to tractable energy models not expressive enough to handle complex high-dimensional objects such as natural images, or to models that do not offer fast writing. We present a novel meta-learning approach to energy-based memory models (EBMM) that allows one to use an arbitrary neural architecture as an energy model and quickly store patterns in its weights. We demonstrate experimentally that our EBMM approach can build compressed memories for synthetic and natural data, and is capable of associative retrieval that outperforms existing memory systems in terms of the reconstruction error and compression rate. |
| Tasks | Meta-Learning |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SyljQyBFDH |
| https://openreview.net/pdf?id=SyljQyBFDH | |
| PWC | https://paperswithcode.com/paper/meta-learning-deep-energy-based-memory-models |
| Repo | |
| Framework | |
Kernel and Rich Regimes in Overparametrized Models
| Title | Kernel and Rich Regimes in Overparametrized Models |
| Authors | Anonymous |
| Abstract | A recent line of work studies overparametrized neural networks in the “kernel regime,” i.e. when the network behaves during training as a kernelized linear predictor, and thus training with gradient descent has the effect of finding the minimum RKHS norm solution. This stands in contrast to other studies which demonstrate how gradient descent on overparametrized multilayer networks can induce rich implicit biases that are not RKHS norms. Building on an observation by Chizat and Bach, we show how the scale of the initialization controls the transition between the “kernel” (aka lazy) and “rich” (aka active) regimes and affects generalization properties in multilayer homogeneous models. We provide a complete and detailed analysis for a simple two-layer model that already exhibits an interesting and meaningful transition between the kernel and rich regimes, and we demonstrate the transition for more complex matrix factorization models and multilayer non-linear networks. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=Byg9bxrtwS |
| https://openreview.net/pdf?id=Byg9bxrtwS | |
| PWC | https://paperswithcode.com/paper/kernel-and-rich-regimes-in-overparametrized |
| Repo | |
| Framework | |
Fast Machine Learning with Byzantine Workers and Servers
| Title | Fast Machine Learning with Byzantine Workers and Servers |
| Authors | Anonymous |
| Abstract | Machine Learning (ML) solutions are nowadays distributed and are prone to various types of component failures, which can be encompassed in so-called Byzantine behavior. This paper introduces LiuBei, a Byzantine-resilient ML algorithm that does not trust any individual component in the network (neither workers nor servers), nor does it induce additional communication rounds (on average), compared to standard non-Byzantine resilient algorithms. LiuBei builds upon gradient aggregation rules (GARs) to tolerate a minority of Byzantine workers. Besides, LiuBei replicates the parameter server on multiple machines instead of trusting it. We introduce a novel filtering mechanism that enables workers to filter out replies from Byzantine server replicas without requiring communication with all servers. Such a filtering mechanism is based on network synchrony, Lipschitz continuity of the loss function, and the GAR used to aggregate workers’ gradients. We also introduce a protocol, scatter/gather, to bound drifts between models on correct servers with a small number of communication messages. We theoretically prove that LiuBei achieves Byzantine resilience to both servers and workers and guarantees convergence. We build LiuBei using TensorFlow, and we show that LiuBei tolerates Byzantine behavior with an accuracy loss of around 5% and around 24% convergence overhead compared to vanilla TensorFlow. We moreover show that the throughput gain of LiuBei compared to another state–of–the–art Byzantine–resilient ML algorithm (that assumes network asynchrony) is 70%. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=B1x996EKPS |
| https://openreview.net/pdf?id=B1x996EKPS | |
| PWC | https://paperswithcode.com/paper/fast-machine-learning-with-byzantine-workers |
| Repo | |
| Framework | |
Fooling Detection Alone is Not Enough: Adversarial Attack against Multiple Object Tracking
| Title | Fooling Detection Alone is Not Enough: Adversarial Attack against Multiple Object Tracking |
| Authors | Yunhan Jia, Yantao Lu, Junjie Shen, Qi Alfred Chen, Hao Chen, Zhenyu Zhong, Tao Wei |
| Abstract | Recent work in adversarial machine learning started to focus on the visual perception in autonomous driving and studied Adversarial Examples (AEs) for object detection models. However, in such visual perception pipeline the detected objects must also be tracked, in a process called Multiple Object Tracking (MOT), to build the moving trajectories of surrounding obstacles. Since MOT is designed to be robust against errors in object detection, it poses a general challenge to existing attack techniques that blindly target objection detection: we find that a success rate of over 98% is needed for them to actually affect the tracking results, a requirement that no existing attack technique can satisfy. In this paper, we are the first to study adversarial machine learning attacks against the complete visual perception pipeline in autonomous driving, and discover a novel attack technique, tracker hijacking, that can effectively fool MOT using AEs on object detection. Using our technique, successful AEs on as few as one single frame can move an existing object in to or out of the headway of an autonomous vehicle to cause potential safety hazards. We perform evaluation using the Berkeley Deep Drive dataset and find that on average when 3 frames are attacked, our attack can have a nearly 100% success rate while attacks that blindly target object detection only have up to 25%. |
| Tasks | Adversarial Attack, Autonomous Driving, Multiple Object Tracking, Object Detection, Object Tracking |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=rJl31TNYPr |
| https://openreview.net/pdf?id=rJl31TNYPr | |
| PWC | https://paperswithcode.com/paper/fooling-detection-alone-is-not-enough |
| Repo | |
| Framework | |
In Search for a SAT-friendly Binarized Neural Network Architecture
| Title | In Search for a SAT-friendly Binarized Neural Network Architecture |
| Authors | Anonymous |
| Abstract | Analyzing the behavior of neural networks is one of the most pressing challenges in deep learning. Binarized Neural Networks are an important class of networks that allow equivalent representation in Boolean logic and can be analyzed formally with logic-based reasoning tools like SAT solvers. Such tools can be used to answer existential and probabilistic queries about the network, perform explanation generation, etc. However, the main bottleneck for all methods is their ability to reason about large BNNs efficiently. In this work, we analyze architectural design choices of BNNs and discuss how they affect the performance of logic-based reasoners. We propose changes to the BNN architecture and the training procedure to get a simpler network for SAT solvers without sacrificing accuracy on the primary task. Our experimental results demonstrate that our approach scales to larger deep neural networks compared to existing work for existential and probabilistic queries, leading to significant speed ups on all tested datasets. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SJx-j64FDr |
| https://openreview.net/pdf?id=SJx-j64FDr | |
| PWC | https://paperswithcode.com/paper/in-search-for-a-sat-friendly-binarized-neural |
| Repo | |
| Framework | |
