April 1, 2020

3290 words 16 mins read

Paper Group ANR 414

On Geometry of Information Flow for Causal Inference. A Survey on Causal Inference. The Counterfactual $χ$-GAN. Vamsa: Tracking Provenance in Data Science Scripts. Night-time Semantic Segmentation with a Large Real Dataset. Tensorized Random Projections. Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?. Adver …

On Geometry of Information Flow for Causal Inference


Title	On Geometry of Information Flow for Causal Inference
Authors	Sudam Surasinghe, Erik M. Bollt
Abstract	Causal inference is perhaps one of the most fundamental concepts in science, beginning originally from the works of some of the ancient philosophers, through today, but also weaved strongly in current work from statisticians, machine learning experts, and scientists from many other fields. This paper takes the perspective of information flow, which includes the Nobel prize winning work on Granger-causality, and the recently highly popular transfer entropy, these being probabilistic in nature. Our main contribution will be to develop analysis tools that will allow a geometric interpretation of information flow as a causal inference indicated by positive transfer entropy. We will describe the effective dimensionality of an underlying manifold as projected into the outcome space that summarizes information flow. Therefore contrasting the probabilistic and geometric perspectives, we will introduce a new measure of causal inference based on the fractal correlation dimension conditionally applied to competing explanations of future forecasts, which we will write $GeoC_{y\rightarrow x}$. This avoids some of the boundedness issues that we show exist for the transfer entropy, $T_{y\rightarrow x}$. We will highlight our discussions with data developed from synthetic models of successively more complex nature: then include the H'{e}non map example, and finally a real physiological example relating breathing and heart rate function. Keywords: Causal Inference; Transfer Entropy; Differential Entropy; Correlation Dimension; Pinsker’s Inequality; Frobenius-Perron operator.
Tasks	Causal Inference
Published	2020-02-06
URL	https://arxiv.org/abs/2002.02078v2
PDF	https://arxiv.org/pdf/2002.02078v2.pdf
PWC	https://paperswithcode.com/paper/on-geometry-of-information-flow-for-causal
Repo
Framework

A Survey on Causal Inference


Title	A Survey on Causal Inference
Authors	Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, Aidong Zhang
Abstract	Causal inference is a critical research topic across many domains, such as statistics, computer science, education, public policy and economics, for decades. Nowadays, estimating causal effect from observational data has become an appealing research direction owing to the large amount of available data and low budget requirement, compared with randomized controlled trials. Embraced with the rapidly developed machine learning area, various causal effect estimation methods for observational data have sprung up. In this survey, we provide a comprehensive review of causal inference methods under the potential outcome framework, one of the well known causal inference framework. The methods are divided into two categories depending on whether they require all three assumptions of the potential outcome framework or not. For each category, both the traditional statistical methods and the recent machine learning enhanced methods are discussed and compared. The plausible applications of these methods are also presented, including the applications in advertising, recommendation, medicine and so on. Moreover, the commonly used benchmark datasets as well as the open-source codes are also summarized, which facilitate researchers and practitioners to explore, evaluate and apply the causal inference methods.
Tasks	Causal Inference
Published	2020-02-05
URL	https://arxiv.org/abs/2002.02770v1
PDF	https://arxiv.org/pdf/2002.02770v1.pdf
PWC	https://paperswithcode.com/paper/a-survey-on-causal-inference
Repo
Framework

The Counterfactual $χ$-GAN


Title	The Counterfactual $χ$-GAN
Authors	Amelia J. Averitt, Natnicha Vanitchanant, Rajesh Ranganath, Adler J. Perotte
Abstract	Causal inference often relies on the counterfactual framework, which requires that treatment assignment is independent of the outcome, known as strong ignorability. Approaches to enforcing strong ignorability in causal analyses of observational data include weighting and matching methods. Effect estimates, such as the average treatment effect (ATE), are then estimated as expectations under the reweighted or matched distribution, P . The choice of P is important and can impact the interpretation of the effect estimate and the variance of effect estimates. In this work, instead of specifying P, we learn a distribution that simultaneously maximizes coverage and minimizes variance of ATE estimates. In order to learn this distribution, this research proposes a generative adversarial network (GAN)-based model called the Counterfactual $\chi$-GAN (cGAN), which also learns feature-balancing weights and supports unbiased causal estimation in the absence of unobserved confounding. Our model minimizes the Pearson $\chi^2$ divergence, which we show simultaneously maximizes coverage and minimizes the variance of importance sampling estimates. To our knowledge, this is the first such application of the Pearson $\chi^2$ divergence. We demonstrate the effectiveness of cGAN in achieving feature balance relative to established weighting methods in simulation and with real-world medical data.
Tasks	Causal Inference
Published	2020-01-09
URL	https://arxiv.org/abs/2001.03115v1
PDF	https://arxiv.org/pdf/2001.03115v1.pdf
PWC	https://paperswithcode.com/paper/the-counterfactual-gan
Repo
Framework

Vamsa: Tracking Provenance in Data Science Scripts


Title	Vamsa: Tracking Provenance in Data Science Scripts
Authors	Mohammad Hossein Namaki, Avrilia Floratou, Fotis Psallidas, Subru Krishnan, Ashvin Agrawal, Yinghui Wu
Abstract	Machine learning (ML) which was initially adopted for search ranking and recommendation systems has firmly moved into the realm of core enterprise operations like sales optimization and preventative healthcare. For such ML applications, often deployed in regulated environments, the standards for user privacy, security, and data governance are substantially higher. This imposes the need for tracking provenance end-to-end, from the data sources used for training ML models to the predictions of the deployed models. In this work, we take a first step towards this direction by introducing the ML provenance tracking problem in the context of data science scripts. The fundamental idea is to automatically identify the relationships between data and ML models and in particular, to track which columns in a dataset have been used to derive the features of a ML model. We discuss the challenges in capturing such provenance information in the context of Python, the most common language used by data scientists. We then, present Vamsa, a modular system that extracts provenance from Python scripts without requiring any changes to the user’s code. Using up to 450K real-world data science scripts from Kaggle and publicly available Python notebooks, we verify the effectiveness of Vamsa in terms of coverage, and performance. We also evaluate Vamsa’s accuracy on a smaller subset of manually labeled data. Our analysis shows that Vamsa’s precision and recall range from 87.5% to 98.3% and its latency is typically in the order of milliseconds for scripts of average size.
Tasks	Recommendation Systems
Published	2020-01-07
URL	https://arxiv.org/abs/2001.01861v1
PDF	https://arxiv.org/pdf/2001.01861v1.pdf
PWC	https://paperswithcode.com/paper/vamsa-tracking-provenance-in-data-science
Repo
Framework

Night-time Semantic Segmentation with a Large Real Dataset


Title	Night-time Semantic Segmentation with a Large Real Dataset
Authors	Xin Tan, Yiheng Zhang, Ying Cao, Lizhuang Ma, Rynson W. H. Lau
Abstract	Although huge progress has been made on semantic segmentation in recent years, most existing works assume that the input images are captured in day-time with good lighting conditions. In this work, we aim to address the semantic segmentation problem of night-time scenes, which has two main challenges: 1) labeled night-time data are scarce, and 2) over- and under-exposures may co-occur in the input night-time images and are not explicitly modeled in existing semantic segmentation pipelines. To tackle the scarcity of night-time data, we collect a novel labeled dataset (named NightCity) of 4,297 real night-time images with ground truth pixel-level semantic annotations. To our knowledge, NightCity is the largest dataset for night-time semantic segmentation. In addition, we also propose an exposure-aware framework to address the night-time segmentation problem through augmenting the segmentation process with explicitly learned exposure features. Extensive experiments show that training on NightCity can significantly improve the performance of night-time semantic segmentation and that our exposure-aware model outperforms the state-of-the-art segmentation methods, yielding top performances on our benchmark dataset.
Tasks	Semantic Segmentation
Published	2020-03-15
URL	https://arxiv.org/abs/2003.06883v1
PDF	https://arxiv.org/pdf/2003.06883v1.pdf
PWC	https://paperswithcode.com/paper/night-time-semantic-segmentation-with-a-large
Repo
Framework

Tensorized Random Projections


Title	Tensorized Random Projections
Authors	Beheshteh T. Rakhshan, Guillaume Rabusseau
Abstract	We introduce a novel random projection technique for efficiently reducing the dimension of very high-dimensional tensors. Building upon classical results on Gaussian random projections and Johnson-Lindenstrauss transforms~(JLT), we propose two tensorized random projection maps relying on the tensor train~(TT) and CP decomposition format, respectively. The two maps offer very low memory requirements and can be applied efficiently when the inputs are low rank tensors given in the CP or TT format. Our theoretical analysis shows that the dense Gaussian matrix in JLT can be replaced by a low-rank tensor implicitly represented in compressed form with random factors, while still approximately preserving the Euclidean distance of the projected inputs. In addition, our results reveal that the TT format is substantially superior to CP in terms of the size of the random projection needed to achieve the same distortion ratio. Experiments on synthetic data validate our theoretical analysis and demonstrate the superiority of the TT decomposition.
Tasks
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05101v1
PDF	https://arxiv.org/pdf/2003.05101v1.pdf
PWC	https://paperswithcode.com/paper/tensorized-random-projections
Repo
Framework

Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?


Title	Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?
Authors	Yixuan Tang, Hwee Tou Ng, Anthony K. H. Tung
Abstract	Multi-hop question answering (QA) requires a model to retrieve and integrate information from different parts of a long text to answer a question. Humans answer this kind of complex questions via a divide-and-conquer approach. In this paper, we investigate whether top-performing models for multi-hop questions understand the underlying sub-questions like humans. We adopt a neural decomposition model to generate sub-questions for a multi-hop complex question, followed by extracting the corresponding sub-answers. We show that multiple state-of-the-art multi-hop QA models fail to correctly answer a large portion of sub-questions, although their corresponding multi-hop questions are correctly answered. This indicates that these models manage to answer the multi-hop questions using some partial clues, instead of truly understanding the reasoning paths. We also propose a new model which significantly improves the performance on answering the sub-questions. Our work takes a step forward towards building a more explainable multi-hop QA system.
Tasks	Question Answering
Published	2020-02-23
URL	https://arxiv.org/abs/2002.09919v1
PDF	https://arxiv.org/pdf/2002.09919v1.pdf
PWC	https://paperswithcode.com/paper/do-multi-hop-question-answering-systems-know
Repo
Framework

Adversarial Policies in Learning Systems with Malicious Experts


Title	Adversarial Policies in Learning Systems with Malicious Experts
Authors	S. Rasoul Etesami, Negar Kiyavash, H. Vincent Poor
Abstract	We consider a learning system based on the conventional multiplicative weight (MW) rule that combines experts’ advice to predict a sequence of true outcomes. It is assumed that one of the experts is malicious and aims to impose the maximum loss on the system. The loss of the system is naturally defined to be the aggregate absolute difference between the sequence of predicted outcomes and the true outcomes. We consider this problem under both offline and online settings. In the offline setting where the malicious expert must choose its entire sequence of decisions a priori, we show somewhat surprisingly that a simple greedy policy of always reporting false prediction is asymptotically optimal with an approximation ratio of $1+O(\sqrt{\frac{\ln N}{N}})$, where $N$ is the total number of prediction stages. In particular, we describe a policy that closely resembles the structure of the optimal offline policy. For the online setting where the malicious expert can adaptively make its decisions, we show that the optimal online policy can be efficiently computed by solving a dynamic program in $O(N^2)$. Our results provide a new direction for vulnerability assessment of commonly used learning algorithms to adversarial attacks where the threat is an integral part of the system.
Tasks
Published	2020-01-02
URL	https://arxiv.org/abs/2001.00543v1
PDF	https://arxiv.org/pdf/2001.00543v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-policies-in-learning-systems-with
Repo
Framework

Bandits with adversarial scaling


Title	Bandits with adversarial scaling
Authors	Thodoris Lykouris, Vahab Mirrokni, Renato Paes Leme
Abstract	We study “adversarial scaling”, a multi-armed bandit model where rewards have a stochastic and an adversarial component. Our model captures display advertising where the “click-through-rate” can be decomposed to a (fixed across time) arm-quality component and a non-stochastic user-relevance component (fixed across arms). Despite the relative stochasticity of our model, we demonstrate two settings where most bandit algorithms suffer. On the positive side, we show that two algorithms, one from the action elimination and one from the mirror descent family are adaptive enough to be robust to adversarial scaling. Our results shed light on the robustness of adaptive parameter selection in stochastic bandits, which may be of independent interest.
Tasks
Published	2020-03-04
URL	https://arxiv.org/abs/2003.02287v1
PDF	https://arxiv.org/pdf/2003.02287v1.pdf
PWC	https://paperswithcode.com/paper/bandits-with-adversarial-scaling
Repo
Framework

Deep Learning in Medical Ultrasound Image Segmentation: a Review


Title	Deep Learning in Medical Ultrasound Image Segmentation: a Review
Authors	Ziyang Wang, Zhengdong Zhang, Jianqing Zheng, Baoru Huang, Irina Voiculescu, Guang-Zhong Yang
Abstract	Applying machine learning technologies, especially deep learning, into medical image segmentation is being widely studied because of its state-of-the-art performance and results. It can be a key step to provide a reliable basis for clinical diagnosis, such as 3D reconstruction of human tissues, image-guided interventions, image analyzing and visualization. In this review article, deep-learning-based methods for ultrasound image segmentation are categorized into six main groups according to their architectures and training at first. Secondly, for each group, several current representative algorithms are selected, introduced, analyzed and summarized in detail. In addition, common evaluation methods for image segmentation and ultrasound image segmentation datasets are summarized. Further, the performance of the current methods and their evaluations are reviewed. In the end, the challenges and potential research directions for medical ultrasound image segmentation are discussed.
Tasks	3D Reconstruction, Medical Image Segmentation, Semantic Segmentation
Published	2020-02-18
URL	https://arxiv.org/abs/2002.07703v2
PDF	https://arxiv.org/pdf/2002.07703v2.pdf
PWC	https://paperswithcode.com/paper/deep-learning-in-medical-ultrasound-image
Repo
Framework

Convolutional Occupancy Networks


Title	Convolutional Occupancy Networks
Authors	Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, Andreas Geiger
Abstract	Recently, implicit neural representations have gained popularity for learning-based 3D reconstruction. While demonstrating promising results, most implicit approaches are limited to comparably simple geometry of single objects and do not scale to more complicated or large-scale scenes. The key limiting factor of implicit methods is their simple fully-connected network architecture which does not allow for integrating local information in the observations or incorporating inductive biases such as translational equivariance. In this paper, we propose Convolutional Occupancy Networks, a more flexible implicit representation for detailed reconstruction of objects and 3D scenes. By combining convolutional encoders with implicit occupancy decoders, our model incorporates inductive biases and Manhattan-world priors, enabling structured reasoning in 3D space. We investigate the effectiveness of the proposed representation by reconstructing complex geometry from noisy point clouds and low-resolution voxel representations. We empirically find that our method enables fine-grained implicit 3D reconstruction of single objects, scales to large indoor scenes and generalizes well from synthetic to real data.
Tasks	3D Reconstruction
Published	2020-03-10
URL	https://arxiv.org/abs/2003.04618v1
PDF	https://arxiv.org/pdf/2003.04618v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-occupancy-networks
Repo
Framework

Learning Extremal Representations with Deep Archetypal Analysis


Title	Learning Extremal Representations with Deep Archetypal Analysis
Authors	Sebastian Mathias Keller, Maxim Samarin, Fabricio Arend Torres, Mario Wieser, Volker Roth
Abstract	Archetypes are typical population representatives in an extremal sense, where typicality is understood as the most extreme manifestation of a trait or feature. In linear feature space, archetypes approximate the data convex hull allowing all data points to be expressed as convex mixtures of archetypes. However, it might not always be possible to identify meaningful archetypes in a given feature space. Learning an appropriate feature space and identifying suitable archetypes simultaneously addresses this problem. This paper introduces a generative formulation of the linear archetype model, parameterized by neural networks. By introducing the distance-dependent archetype loss, the linear archetype model can be integrated into the latent space of a variational autoencoder, and an optimal representation with respect to the unknown archetypes can be learned end-to-end. The reformulation of linear Archetypal Analysis as deep variational information bottleneck, allows the incorporation of arbitrarily complex side information during training. Furthermore, an alternative prior, based on a modified Dirichlet distribution, is proposed. The real-world applicability of the proposed method is demonstrated by exploring archetypes of female facial expressions while using multi-rater based emotion scores of these expressions as side information. A second application illustrates the exploration of the chemical space of small organic molecules. In this experiment, it is demonstrated that exchanging the side information but keeping the same set of molecules, e. g. using as side information the heat capacity of each molecule instead of the band gap energy, will result in the identification of different archetypes. As an application, these learned representations of chemical space might reveal distinct starting points for de novo molecular design.
Tasks	Band Gap
Published	2020-02-03
URL	https://arxiv.org/abs/2002.00815v1
PDF	https://arxiv.org/pdf/2002.00815v1.pdf
PWC	https://paperswithcode.com/paper/learning-extremal-representations-with-deep
Repo
Framework

Deep convolutional embedding for digitized painting clustering


Title	Deep convolutional embedding for digitized painting clustering
Authors	Giovanna Castellano, Gennaro Vessio
Abstract	Clustering artworks is difficult because of several reasons. On one hand, recognizing meaningful patterns in accordance with domain knowledge and visual perception is extremely hard. On the other hand, the application of traditional clustering and feature reduction techniques to the highly dimensional pixel space can be ineffective. To address these issues, we propose a deep convolutional embedding model for clustering digital paintings, in which the task of mapping the input raw data to an abstract, latent space is optimized jointly with the task of finding a set of cluster centroids in this latent feature space. Quantitative and qualitative experimental results show the effectiveness of the proposed method. The model is also able to outperform other state-of-the-art deep clustering approaches to the same problem. The proposed method may be beneficial to several art-related tasks, particularly visual link retrieval and historical knowledge discovery in painting datasets.
Tasks
Published	2020-03-19
URL	https://arxiv.org/abs/2003.08597v1
PDF	https://arxiv.org/pdf/2003.08597v1.pdf
PWC	https://paperswithcode.com/paper/deep-convolutional-embedding-for-digitized
Repo
Framework

Active Perception with A Monocular Camera for Multiscopic Vision


Title	Active Perception with A Monocular Camera for Multiscopic Vision
Authors	Weihao Yuan, Rui Fan, Michael Yu Wang, Qifeng Chen
Abstract	We design a multiscopic vision system that utilizes a low-cost monocular RGB camera to acquire accurate depth estimation for robotic applications. Unlike multi-view stereo with images captured at unconstrained camera poses, the proposed system actively controls a robot arm with a mounted camera to capture a sequence of images in horizontally or vertically aligned positions with the same parallax. In this system, we combine the cost volumes for stereo matching between the reference image and the surrounding images to form a fused cost volume that is robust to outliers. Experiments on the Middlebury dataset and real robot experiments show that our obtained disparity maps are more accurate than two-frame stereo matching: the average absolute error is reduced by 50.2% in our experiments.
Tasks	Depth Estimation, Stereo Matching
Published	2020-01-22
URL	https://arxiv.org/abs/2001.08212v1
PDF	https://arxiv.org/pdf/2001.08212v1.pdf
PWC	https://paperswithcode.com/paper/active-perception-with-a-monocular-camera-for
Repo
Framework

PDANet: Pyramid Density-aware Attention Net for Accurate Crowd Counting


Title	PDANet: Pyramid Density-aware Attention Net for Accurate Crowd Counting
Authors	Saeed Amirgholipour, Xiangjian He, Wenjing Jia, Dadong Wang, Lei Liu
Abstract	Crowd counting, i.e., estimating the number of people in a crowded area, has attracted much interest in the research community. Although many attempts have been reported, crowd counting remains an open real-world problem due to the vast scale variations in crowd density within the interested area, and severe occlusion among the crowd. In this paper, we propose a novel Pyramid Density-Aware Attention-based network, abbreviated as PDANet, that leverages the attention, pyramid scale feature and two branch decoder modules for density-aware crowd counting. The PDANet utilizes these modules to extract different scale features, focus on the relevant information, and suppress the misleading ones. We also address the variation of crowdedness levels among different images with an exclusive Density-Aware Decoder (DAD). For this purpose, a classifier evaluates the density level of the input features and then passes them to the corresponding high and low crowded DAD modules. Finally, we generate an overall density map by considering the summation of low and high crowded density maps as spatial attention. Meanwhile, we employ two losses to create a precise density map for the input scene. Extensive evaluations conducted on the challenging benchmark datasets well demonstrate the superior performance of the proposed PDANet in terms of the accuracy of counting and generated density maps over the well-known state of the arts.
Tasks	Crowd Counting
Published	2020-01-16
URL	https://arxiv.org/abs/2001.05643v7
PDF	https://arxiv.org/pdf/2001.05643v7.pdf
PWC	https://paperswithcode.com/paper/pdanet-pyramid-density-aware-attention-net
Repo
Framework