October 19, 2019

3042 words 15 mins read

Paper Group ANR 401

Reducing Gender Bias in Abusive Language Detection. MDU-Net: Multi-scale Densely Connected U-Net for biomedical image segmentation. Distributionally Robust Graphical Models. Data Science with Vadalog: Bridging Machine Learning and Reasoning. From Hard to Soft: Understanding Deep Network Nonlinearities via Vector Quantization and Statistical Inferen …

Reducing Gender Bias in Abusive Language Detection


Title	Reducing Gender Bias in Abusive Language Detection
Authors	Ji Ho Park, Jamin Shin, Pascale Fung
Abstract	Abusive language detection models tend to have a problem of being biased toward identity words of a certain group of people because of imbalanced training datasets. For example, “You are a good woman” was considered “sexist” when trained on an existing dataset. Such model bias is an obstacle for models to be robust enough for practical use. In this work, we measure gender biases on models trained with different abusive language datasets, while analyzing the effect of different pre-trained word embeddings and model architectures. We also experiment with three bias mitigation methods: (1) debiased word embeddings, (2) gender swap data augmentation, and (3) fine-tuning with a larger corpus. These methods can effectively reduce gender bias by 90-98% and can be extended to correct model bias in other scenarios.
Tasks	Data Augmentation, Word Embeddings
Published	2018-08-22
URL	http://arxiv.org/abs/1808.07231v1
PDF	http://arxiv.org/pdf/1808.07231v1.pdf
PWC	https://paperswithcode.com/paper/reducing-gender-bias-in-abusive-language
Repo
Framework

MDU-Net: Multi-scale Densely Connected U-Net for biomedical image segmentation


Title	MDU-Net: Multi-scale Densely Connected U-Net for biomedical image segmentation
Authors	Jiawei Zhang, Yuzhen Jin, Jilan Xu, Xiaowei Xu, Yanchun Zhang
Abstract	Radiologist is “doctor’s doctor”, biomedical image segmentation plays a central role in quantitative analysis, clinical diagnosis, and medical intervention. In the light of the fully convolutional networks (FCN) and U-Net, deep convolutional networks (DNNs) have made significant contributions in biomedical image segmentation applications. In this paper, based on U-Net, we propose MDUnet, a multi-scale densely connected U-net for biomedical image segmentation. we propose three different multi-scale dense connections for U shaped architectures encoder, decoder and across them. The highlights of our architecture is directly fuses the neighboring different scale feature maps from both higher layers and lower layers to strengthen feature propagation in current layer. Which can largely improves the information flow encoder, decoder and across them. Multi-scale dense connections, which means containing shorter connections between layers close to the input and output, also makes much deeper U-net possible. We adopt the optimal model based on the experiment and propose a novel Multi-scale Dense U-Net (MDU-Net) architecture with quantization. Which reduce overfitting in MDU-Net for better accuracy. We evaluate our purpose model on the MICCAI 2015 Gland Segmentation dataset (GlaS). The three multi-scale dense connections improve U-net performance by up to 1.8% on test A and 3.5% on test B in the MICCAI Gland dataset. Meanwhile the MDU-net with quantization achieves the superiority over U-Net performance by up to 3% on test A and 4.1% on test B.
Tasks	Quantization, Semantic Segmentation
Published	2018-12-02
URL	http://arxiv.org/abs/1812.00352v2
PDF	http://arxiv.org/pdf/1812.00352v2.pdf
PWC	https://paperswithcode.com/paper/mdu-net-multi-scale-densely-connected-u-net
Repo
Framework

Distributionally Robust Graphical Models


Title	Distributionally Robust Graphical Models
Authors	Rizal Fathony, Ashkan Rezaei, Mohammad Ali Bashiri, Xinhua Zhang, Brian D. Ziebart
Abstract	In many structured prediction problems, complex relationships between variables are compactly defined using graphical structures. The most prevalent graphical prediction methods—probabilistic graphical models and large margin methods—have their own distinct strengths but also possess significant drawbacks. Conditional random fields (CRFs) are Fisher consistent, but they do not permit integration of customized loss metrics into their learning process. Large-margin models, such as structured support vector machines (SSVMs), have the flexibility to incorporate customized loss metrics, but lack Fisher consistency guarantees. We present adversarial graphical models (AGM), a distributionally robust approach for constructing a predictor that performs robustly for a class of data distributions defined using a graphical structure. Our approach enjoys both the flexibility of incorporating customized loss metrics into its design as well as the statistical guarantee of Fisher consistency. We present exact learning and prediction algorithms for AGM with time complexity similar to existing graphical models and show the practical benefits of our approach with experiments.
Tasks	Structured Prediction
Published	2018-11-07
URL	http://arxiv.org/abs/1811.02728v1
PDF	http://arxiv.org/pdf/1811.02728v1.pdf
PWC	https://paperswithcode.com/paper/distributionally-robust-graphical-models
Repo
Framework

Data Science with Vadalog: Bridging Machine Learning and Reasoning


Title	Data Science with Vadalog: Bridging Machine Learning and Reasoning
Authors	Luigi Bellomarini, Ruslan R. Fayzrakhmanov, Georg Gottlob, Andrey Kravchenko, Eleonora Laurenza, Yavor Nenov, Stephane Reissfelder, Emanuel Sallinger, Evgeny Sherkhonov, Lianlong Wu
Abstract	Following the recent successful examples of large technology companies, many modern enterprises seek to build knowledge graphs to provide a unified view of corporate knowledge and to draw deep insights using machine learning and logical reasoning. There is currently a perceived disconnect between the traditional approaches for data science, typically based on machine learning and statistical modelling, and systems for reasoning with domain knowledge. In this paper we present a state-of-the-art Knowledge Graph Management System, Vadalog, which delivers highly expressive and efficient logical reasoning and provides seamless integration with modern data science toolkits, such as the Jupyter platform. We demonstrate how to use Vadalog to perform traditional data wrangling tasks, as well as complex logical and probabilistic reasoning. We argue that this is a significant step forward towards combining machine learning and reasoning in data science.
Tasks	Knowledge Graphs
Published	2018-07-23
URL	http://arxiv.org/abs/1807.08712v1
PDF	http://arxiv.org/pdf/1807.08712v1.pdf
PWC	https://paperswithcode.com/paper/data-science-with-vadalog-bridging-machine
Repo
Framework

From Hard to Soft: Understanding Deep Network Nonlinearities via Vector Quantization and Statistical Inference


Title	From Hard to Soft: Understanding Deep Network Nonlinearities via Vector Quantization and Statistical Inference
Authors	Randall Balestriero, Richard G. Baraniuk
Abstract	Nonlinearity is crucial to the performance of a deep (neural) network (DN). To date there has been little progress understanding the menagerie of available nonlinearities, but recently progress has been made on understanding the r^ole played by piecewise affine and convex nonlinearities like the ReLU and absolute value activation functions and max-pooling. In particular, DN layers constructed from these operations can be interpreted as {\em max-affine spline operators} (MASOs) that have an elegant link to vector quantization (VQ) and $K$-means. While this is good theoretical progress, the entire MASO approach is predicated on the requirement that the nonlinearities be piecewise affine and convex, which precludes important activation functions like the sigmoid, hyperbolic tangent, and softmax. {\em This paper extends the MASO framework to these and an infinitely large class of new nonlinearities by linking deterministic MASOs with probabilistic Gaussian Mixture Models (GMMs).} We show that, under a GMM, piecewise affine, convex nonlinearities like ReLU, absolute value, and max-pooling can be interpreted as solutions to certain natural “hard” VQ inference problems, while sigmoid, hyperbolic tangent, and softmax can be interpreted as solutions to corresponding “soft” VQ inference problems. We further extend the framework by hybridizing the hard and soft VQ optimizations to create a $\beta$-VQ inference that interpolates between hard, soft, and linear VQ inference. A prime example of a $\beta$-VQ DN nonlinearity is the {\em swish} nonlinearity, which offers state-of-the-art performance in a range of computer vision tasks but was developed ad hoc by experimentation. Finally, we validate with experiments an important assertion of our theory, namely that DN performance can be significantly improved by enforcing orthogonality in its linear filters.
Tasks	Quantization
Published	2018-10-22
URL	http://arxiv.org/abs/1810.09274v1
PDF	http://arxiv.org/pdf/1810.09274v1.pdf
PWC	https://paperswithcode.com/paper/from-hard-to-soft-understanding-deep-network
Repo
Framework

Sum decomposition of divergence into three divergences


Title	Sum decomposition of divergence into three divergences
Authors	Tomohiro Nishiyama
Abstract	Divergence functions play a key role as to measure the discrepancy between two points in the field of machine learning, statistics and signal processing. Well-known divergences are the Bregman divergences, the Jensen divergences and the f-divergences. In this paper, we show that the symmetric Bregman divergence can be decomposed into the sum of two types of Jensen divergences and the Bregman divergence. Furthermore, applying this result, we show another sum decomposition of divergence is possible which includes f-divergences explicitly.
Tasks
Published	2018-10-03
URL	http://arxiv.org/abs/1810.01720v2
PDF	http://arxiv.org/pdf/1810.01720v2.pdf
PWC	https://paperswithcode.com/paper/sum-decomposition-of-divergence-into-three
Repo
Framework

Improved Deep Embeddings for Inferencing with Multi-Layered Networks


Title	Improved Deep Embeddings for Inferencing with Multi-Layered Networks
Authors	Huan Song, Jayaraman J. Thiagarajan
Abstract	Inferencing with network data necessitates the mapping of its nodes into a vector space, where the relationships are preserved. However, with multi-layered networks, where multiple types of relationships exist for the same set of nodes, it is crucial to exploit the information shared between layers, in addition to the distinct aspects of each layer. In this paper, we propose a novel approach that first obtains node embeddings in all layers jointly via DeepWalk on a \textit{supra} graph, which allows interactions between layers, and then fine-tunes the embeddings to encourage cohesive structure in the latent space. With empirical studies in node classification, link prediction and multi-layered community detection, we show that the proposed approach outperforms existing single- and multi-layered network embedding algorithms on several benchmarks. In addition to effectively scaling to a large number of layers (tested up to $37$), our approach consistently produces highly modular community structure, even when compared to methods that directly optimize for the modularity function.
Tasks	Community Detection, Link Prediction, Network Embedding, Node Classification
Published	2018-09-20
URL	http://arxiv.org/abs/1811.12156v2
PDF	http://arxiv.org/pdf/1811.12156v2.pdf
PWC	https://paperswithcode.com/paper/improved-deep-embeddings-for-inferencing-with
Repo
Framework

Improving DNN-based Music Source Separation using Phase Features


Title	Improving DNN-based Music Source Separation using Phase Features
Authors	Joachim Muth, Stefan Uhlich, Nathanael Perraudin, Thomas Kemp, Fabien Cardinaux, Yuki Mitsufuji
Abstract	Music source separation with deep neural networks typically relies only on amplitude features. In this paper we show that additional phase features can improve the separation performance. Using the theoretical relationship between STFT phase and amplitude, we conjecture that derivatives of the phase are a good feature representation opposed to the raw phase. We verify this conjecture experimentally and propose a new DNN architecture which combines amplitude and phase. This joint approach achieves a better signal-to distortion ratio on the DSD100 dataset for all instruments compared to a network that uses only amplitude features. Especially, the bass instrument benefits from the phase information.
Tasks	Music Source Separation
Published	2018-07-07
URL	http://arxiv.org/abs/1807.02710v3
PDF	http://arxiv.org/pdf/1807.02710v3.pdf
PWC	https://paperswithcode.com/paper/improving-dnn-based-music-source-separation
Repo
Framework

Denoising Auto-encoder with Recurrent Skip Connections and Residual Regression for Music Source Separation


Title	Denoising Auto-encoder with Recurrent Skip Connections and Residual Regression for Music Source Separation
Authors	Jen-Yu Liu, Yi-Hsuan Yang
Abstract	Convolutional neural networks with skip connections have shown good performance in music source separation. In this work, we propose a denoising Auto-encoder with Recurrent skip Connections (ARC). We use 1D convolution along the temporal axis of the time-frequency feature map in all layers of the fully-convolutional network. The use of 1D convolution makes it possible to apply recurrent layers to the intermediate outputs of the convolution layers. In addition, we also propose an enhancement network and a residual regression method to further improve the separation result. The recurrent skip connections, the enhancement module, and the residual regression all improve the separation quality. The ARC model with residual regression achieves 5.74 siganl-to-distoration ratio (SDR) in vocals with MUSDB in SiSEC 2018. We also evaluate the ARC model alone on the older dataset DSD100 (used in SiSEC 2016) and it achieves 5.91 SDR in vocals.
Tasks	Denoising, Music Source Separation
Published	2018-07-05
URL	http://arxiv.org/abs/1807.01898v1
PDF	http://arxiv.org/pdf/1807.01898v1.pdf
PWC	https://paperswithcode.com/paper/denoising-auto-encoder-with-recurrent-skip
Repo
Framework

Target Aware Network Adaptation for Efficient Representation Learning


Title	Target Aware Network Adaptation for Efficient Representation Learning
Authors	Yang Zhong, Vladimir Li, Ryuzo Okada, Atsuto Maki
Abstract	This paper presents an automatic network adaptation method that finds a ConvNet structure well-suited to a given target task, e.g., image classification, for efficiency as well as accuracy in transfer learning. We call the concept target-aware transfer learning. Given only small-scale labeled data, and starting from an ImageNet pre-trained network, we exploit a scheme of removing its potential redundancy for the target task through iterative operations of filter-wise pruning and network optimization. The basic motivation is that compact networks are on one hand more efficient and should also be more tolerant, being less complex, against the risk of overfitting which would hinder the generalization of learned representations in the context of transfer learning. Further, unlike existing methods involving network simplification, we also let the scheme identify redundant portions across the entire network, which automatically results in a network structure adapted to the task at hand. We achieve this with a few novel ideas: (i) cumulative sum of activation statistics for each layer, and (ii) a priority evaluation of pruning across multiple layers. Experimental results by the method on five datasets (Flower102, CUB200-2011, Dog120, MIT67, and Stanford40) show favorable accuracies over the related state-of-the-art techniques while enhancing the computational and storage efficiency of the transferred model.
Tasks	Image Classification, Representation Learning, Transfer Learning
Published	2018-10-02
URL	http://arxiv.org/abs/1810.01104v1
PDF	http://arxiv.org/pdf/1810.01104v1.pdf
PWC	https://paperswithcode.com/paper/target-aware-network-adaptation-for-efficient
Repo
Framework

Event-Based Structured Light for Depth Reconstruction using Frequency Tagged Light Patterns


Title	Event-Based Structured Light for Depth Reconstruction using Frequency Tagged Light Patterns
Authors	T. Leroux, S. -H. Ieng, R. Benosman
Abstract	This paper presents a new method for 3D depth estimation using the output of an asynchronous time driven image sensor. In association with a high speed Digital Light Processing projection system, our method achieves real-time reconstruction of 3D points cloud, up to several hundreds of hertz. Unlike state of the art methodology, we introduce a method that relies on the use of frequency tagged light pattern that make use of the high temporal resolution of event based sensors. This approch eases matching as each pattern unique frequency allow for any easy matching between displayed patterns and the event based sensor. Results are show on real scenes.
Tasks	3D Depth Estimation, Depth Estimation
Published	2018-11-27
URL	http://arxiv.org/abs/1811.10771v1
PDF	http://arxiv.org/pdf/1811.10771v1.pdf
PWC	https://paperswithcode.com/paper/event-based-structured-light-for-depth
Repo
Framework

Introduction to the 1st Place Winning Model of OpenImages Relationship Detection Challenge


Title	Introduction to the 1st Place Winning Model of OpenImages Relationship Detection Challenge
Authors	Ji Zhang, Kevin Shih, Andrew Tao, Bryan Catanzaro, Ahmed Elgammal
Abstract	This article describes the model we built that achieved 1st place in the OpenImage Visual Relationship Detection Challenge on Kaggle. Three key factors contribute the most to our success: 1) language bias is a powerful baseline for this task. We build the empirical distribution $P(predicatesubject,object)$ in the training set and directly use that in testing. This baseline achieved the 2nd place when submitted; 2) spatial features are as important as visual features, especially for spatial relationships such as “under” and “inside of”; 3) It is a very effective way to fuse different features by first building separate modules for each of them, then adding their output logits before the final softmax layer. We show in ablation study that each factor can improve the performance to a non-trivial extent, and the model reaches optimal when all of them are combined.
Tasks
Published	2018-11-01
URL	http://arxiv.org/abs/1811.00662v2
PDF	http://arxiv.org/pdf/1811.00662v2.pdf
PWC	https://paperswithcode.com/paper/introduction-to-the-1st-place-winning-model
Repo
Framework

Attentive Tensor Product Learning


Title	Attentive Tensor Product Learning
Authors	Qiuyuan Huang, Li Deng, Dapeng Wu, Chang Liu, Xiaodong He
Abstract	This paper proposes a new architecture - Attentive Tensor Product Learning (ATPL) - to represent grammatical structures in deep learning models. ATPL is a new architecture to bridge this gap by exploiting Tensor Product Representations (TPR), a structured neural-symbolic model developed in cognitive science, aiming to integrate deep learning with explicit language structures and rules. The key ideas of ATPL are: 1) unsupervised learning of role-unbinding vectors of words via TPR-based deep neural network; 2) employing attention modules to compute TPR; and 3) integration of TPR with typical deep learning architectures including Long Short-Term Memory (LSTM) and Feedforward Neural Network (FFNN). The novelty of our approach lies in its ability to extract the grammatical structure of a sentence by using role-unbinding vectors, which are obtained in an unsupervised manner. This ATPL approach is applied to 1) image captioning, 2) part of speech (POS) tagging, and 3) constituency parsing of a sentence. Experimental results demonstrate the effectiveness of the proposed approach.
Tasks	Constituency Parsing, Image Captioning, Part-Of-Speech Tagging
Published	2018-02-20
URL	http://arxiv.org/abs/1802.07089v2
PDF	http://arxiv.org/pdf/1802.07089v2.pdf
PWC	https://paperswithcode.com/paper/attentive-tensor-product-learning
Repo
Framework

DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech


Title	DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech
Authors	George Christodoulides, Mathieu Avanzi, Jean-Philippe Goldman
Abstract	We present DisMo, a multi-level annotator for spoken language corpora that integrates part-of-speech tagging with basic disfluency detection and annotation, and multi-word unit recognition. DisMo is a hybrid system that uses a combination of lexical resources, rules, and statistical models based on Conditional Random Fields (CRF). In this paper, we present the first public version of DisMo for French. The system is trained and its performance evaluated on a 57k-token corpus, including different varieties of French spoken in three countries (Belgium, France and Switzerland). DisMo supports a multi-level annotation scheme, in which the tokenisation to minimal word units is complemented with multi-word unit groupings (each having associated POS tags), as well as separate levels for annotating disfluencies and discourse phenomena. We present the system’s architecture, linguistic resources and its hierarchical tag-set. Results show that DisMo achieves a precision of 95% (finest tag-set) to 96.8% (coarse tag-set) in POS-tagging non-punctuated, sound-aligned transcriptions of spoken French, while also offering substantial possibilities for automated multi-level annotation.
Tasks	Part-Of-Speech Tagging
Published	2018-02-08
URL	http://arxiv.org/abs/1802.02926v1
PDF	http://arxiv.org/pdf/1802.02926v1.pdf
PWC	https://paperswithcode.com/paper/dismo-a-morphosyntactic-disfluency-and-multi
Repo
Framework

Actor-Critic Policy Optimization in Partially Observable Multiagent Environments


Title	Actor-Critic Policy Optimization in Partially Observable Multiagent Environments
Authors	Sriram Srinivasan, Marc Lanctot, Vinicius Zambaldi, Julien Perolat, Karl Tuyls, Remi Munos, Michael Bowling
Abstract	Optimization of parameterized policies for reinforcement learning (RL) is an important and challenging problem in artificial intelligence. Among the most common approaches are algorithms based on gradient ascent of a score function representing discounted return. In this paper, we examine the role of these policy gradient and actor-critic algorithms in partially-observable multiagent environments. We show several candidate policy update rules and relate them to a foundation of regret minimization and multiagent learning techniques for the one-shot and tabular cases, leading to previously unknown convergence guarantees. We apply our method to model-free multiagent reinforcement learning in adversarial sequential decision problems (zero-sum imperfect information games), using RL-style function approximation. We evaluate on commonly used benchmark Poker domains, showing performance against fixed policies and empirical convergence to approximate Nash equilibria in self-play with rates similar to or better than a baseline model-free algorithm for zero sum games, without any domain-specific state space reductions.
Tasks
Published	2018-10-21
URL	https://arxiv.org/abs/1810.09026v4
PDF	https://arxiv.org/pdf/1810.09026v4.pdf
PWC	https://paperswithcode.com/paper/actor-critic-policy-optimization-in-partially
Repo
Framework