January 29, 2020

2956 words 14 mins read

Paper Group ANR 710

A Review, Framework and R toolkit for Exploring, Evaluating, and Comparing Visualizations. Multi-task Pairwise Neural Ranking for Hashtag Segmentation. In-domain representation learning for remote sensing. Visualization of AE’s Training on Credit Card Transactions with Persistent Homology. Unraveling the Veil of Subspace RIP Through Near-Isometry o …

A Review, Framework and R toolkit for Exploring, Evaluating, and Comparing Visualizations


Title	A Review, Framework and R toolkit for Exploring, Evaluating, and Comparing Visualizations
Authors	Stephen L. France, Ulas Akkucuk
Abstract	This paper gives a review and synthesis of methods of evaluating dimensionality reduction techniques. Particular attention is paid to rank-order neighborhood evaluation metrics. A framework is created for exploring dimensionality reduction quality through visualization. An associated toolkit is implemented in R. The toolkit includes scatter plots, heat maps, loess smoothing, and performance lift diagrams. The overall rationale is to help researchers compare dimensionality reduction techniques and use visual insights to help select and improve techniques. Examples are given for dimensionality reduction of manifolds and for the dimensionality reduction applied to a consumer survey dataset.
Tasks	Dimensionality Reduction
Published	2019-02-22
URL	http://arxiv.org/abs/1902.08571v1
PDF	http://arxiv.org/pdf/1902.08571v1.pdf
PWC	https://paperswithcode.com/paper/a-review-framework-and-r-toolkit-for
Repo
Framework

Multi-task Pairwise Neural Ranking for Hashtag Segmentation


Title	Multi-task Pairwise Neural Ranking for Hashtag Segmentation
Authors	Mounica Maddela, Wei Xu, Daniel Preoţiuc-Pietro
Abstract	Hashtags are often employed on social media and beyond to add metadata to a textual utterance with the goal of increasing discoverability, aiding search, or providing additional semantics. However, the semantic content of hashtags is not straightforward to infer as these represent ad-hoc conventions which frequently include multiple words joined together and can include abbreviations and unorthodox spellings. We build a dataset of 12,594 hashtags split into individual segments and propose a set of approaches for hashtag segmentation by framing it as a pairwise ranking problem between candidate segmentations. Our novel neural approaches demonstrate 24.6% error reduction in hashtag segmentation accuracy compared to the current state-of-the-art method. Finally, we demonstrate that a deeper understanding of hashtag semantics obtained through segmentation is useful for downstream applications such as sentiment analysis, for which we achieved a 2.6% increase in average recall on the SemEval 2017 sentiment analysis dataset.
Tasks	Sentiment Analysis
Published	2019-06-03
URL	https://arxiv.org/abs/1906.00790v2
PDF	https://arxiv.org/pdf/1906.00790v2.pdf
PWC	https://paperswithcode.com/paper/190600790
Repo
Framework

In-domain representation learning for remote sensing


Title	In-domain representation learning for remote sensing
Authors	Maxim Neumann, Andre Susano Pinto, Xiaohua Zhai, Neil Houlsby
Abstract	Given the importance of remote sensing, surprisingly little attention has been paid to it by the representation learning community. To address it and to establish baselines and a common evaluation protocol in this domain, we provide simplified access to 5 diverse remote sensing datasets in a standardized form. Specifically, we investigate in-domain representation learning to develop generic remote sensing representations and explore which characteristics are important for a dataset to be a good source for remote sensing representation learning. The established baselines achieve state-of-the-art performance on these datasets.
Tasks	Representation Learning
Published	2019-11-15
URL	https://arxiv.org/abs/1911.06721v1
PDF	https://arxiv.org/pdf/1911.06721v1.pdf
PWC	https://paperswithcode.com/paper/in-domain-representation-learning-for-remote-1
Repo
Framework

Visualization of AE’s Training on Credit Card Transactions with Persistent Homology


Title	Visualization of AE’s Training on Credit Card Transactions with Persistent Homology
Authors	Jeremy Charlier, Francois Petit, Gaston Ormazabal, Radu State, Jean Hilger
Abstract	Auto-encoders are among the most popular neural network architecture for dimension reduction. They are composed of two parts: the encoder which maps the model distribution to a latent manifold and the decoder which maps the latent manifold to a reconstructed distribution. However, auto-encoders are known to provoke chaotically scattered data distribution in the latent manifold resulting in an incomplete reconstructed distribution. Current distance measures fail to detect this problem because they are not able to acknowledge the shape of the data manifolds, i.e. their topological features, and the scale at which the manifolds should be analyzed. We propose Persistent Homology for Wasserstein Auto-Encoders, called PHom-WAE, a new methodology to assess and measure the data distribution of a generative model. PHom-WAE minimizes the Wasserstein distance between the true distribution and the reconstructed distribution and uses persistent homology, the study of the topological features of a space at different spatial resolutions, to compare the nature of the latent manifold and the reconstructed distribution. Our experiments underline the potential of persistent homology for Wasserstein Auto-Encoders in comparison to Variational Auto-Encoders, another type of generative model. The experiments are conducted on a real-world data set particularly challenging for traditional distance measures and auto-encoders. PHom-WAE is the first methodology to propose a topological distance measure, the bottleneck distance, for Wasserstein Auto-Encoders used to compare decoded samples of high quality in the context of credit card transactions.
Tasks	Dimensionality Reduction
Published	2019-05-24
URL	https://arxiv.org/abs/1905.13020v2
PDF	https://arxiv.org/pdf/1905.13020v2.pdf
PWC	https://paperswithcode.com/paper/190513020
Repo
Framework

Unraveling the Veil of Subspace RIP Through Near-Isometry on Subspaces


Title	Unraveling the Veil of Subspace RIP Through Near-Isometry on Subspaces
Authors	Xingyu Xv, Gen Li, Yuantao Gu
Abstract	Dimensionality reduction is a popular approach to tackle high-dimensional data with low-dimensional nature. Subspace Restricted Isometry Property, a newly-proposed concept, has proved to be a useful tool in analyzing the effect of dimensionality reduction algorithms on subspaces. In this paper, we provide a characterization of subspace Restricted Isometry Property, asserting that matrices which act as a near-isometry on low-dimensional subspaces possess subspace Restricted Isometry Property. This points out a unified approach to discuss subspace Restricted Isometry Property. Its power is further demonstrated by the possibility to prove with this result the subspace RIP for a large variety of random matrices encountered in theory and practice, including subgaussian matrices, partial Fourier matrices, partial Hadamard matrices, partial circulant/Toeplitz matrices, matrices with independent strongly regular rows (for instance, matrices with independent entries having uniformly bounded $4+\epsilon$ moments), and log-concave ensembles. Thus our result could extend the applicability of random projections in subspace-based machine learning algorithms including subspace clustering and allow for the application of some useful random matrices which are easier to implement on hardware or are more efficient to compute.
Tasks	Dimensionality Reduction
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09608v2
PDF	https://arxiv.org/pdf/1905.09608v2.pdf
PWC	https://paperswithcode.com/paper/johnson-lindenstrauss-property-implies
Repo
Framework

CASS: Cross Adversarial Source Separation via Autoencoder


Title	CASS: Cross Adversarial Source Separation via Autoencoder
Authors	Yong Zheng Ong, Charles K. Chui, Haizhao Yang
Abstract	This paper introduces a cross adversarial source separation (CASS) framework via autoencoder, a new model that aims at separating an input signal consisting of a mixture of multiple components into individual components defined via adversarial learning and autoencoder fitting. CASS unifies popular generative networks like auto-encoders (AEs) and generative adversarial networks (GANs) in a single framework. The basic building block that filters the input signal and reconstructs the $i$-th target component is a pair of deep neural networks $\mathcal{EN}_i$ and $\mathcal{DE}_i$ as an encoder for dimension reduction and a decoder for component reconstruction, respectively. The decoder $\mathcal{DE}_i$ as a generator is enhanced by a discriminator network $\mathcal{D}_i$ that favors signal structures of the $i$-th component in the $i$-th given dataset as guidance through adversarial learning. In contrast with existing practices in AEs which trains each Auto-Encoder independently, or in GANs that share the same generator, we introduce cross adversarial training that emphasizes adversarial relation between any arbitrary network pairs $(\mathcal{DE}_i,\mathcal{D}_j)$, achieving state-of-the-art performance especially when target components share similar data structures.
Tasks	Dimensionality Reduction
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09877v1
PDF	https://arxiv.org/pdf/1905.09877v1.pdf
PWC	https://paperswithcode.com/paper/cass-cross-adversarial-source-separation-via
Repo
Framework

Defining Image Memorability using the Visual Memory Schema


Title	Defining Image Memorability using the Visual Memory Schema
Authors	Erdem Akagunduz, Adrian G. Bors, Karla K. Evans
Abstract	Memorability of an image is a characteristic determined by the human observers’ ability to remember images they have seen. Yet recent work on image memorability defines it as an intrinsic property that can be obtained independent of the observer. {The current study aims to enhance our understanding and prediction of image memorability, improving upon existing approaches by incorporating the properties of cumulative human annotations.} We propose a new concept called the Visual Memory Schema (VMS) referring to an organisation of image components human observers share when encoding and recognising images. The concept of VMS is operationalised by asking human observers to define memorable regions of images they were asked to remember during an episodic memory test. We then statistically assess the consistency of VMSs across observers for either correctly or incorrectly recognised images. The associations of the VMSs with eye fixations and saliency are analysed separately as well. Lastly, we adapt various deep learning architectures for the reconstruction and prediction of memorable regions in images and analyse the results when using transfer learning at the outputs of different convolutional network layers.
Tasks	Transfer Learning
Published	2019-03-05
URL	http://arxiv.org/abs/1903.02056v1
PDF	http://arxiv.org/pdf/1903.02056v1.pdf
PWC	https://paperswithcode.com/paper/defining-image-memorability-using-the-visual
Repo
Framework

Sentence Embeddings for Russian NLU


Title	Sentence Embeddings for Russian NLU
Authors	Dmitry Popov, Alexander Pugachev, Polina Svyatokum, Elizaveta Svitanko, Ekaterina Artemova
Abstract	We investigate the performance of sentence embeddings models on several tasks for the Russian language. In our comparison, we include such tasks as multiple choice question answering, next sentence prediction, and paraphrase identification. We employ FastText embeddings as a baseline and compare it to ELMo and BERT embeddings. We conduct two series of experiments, using both unsupervised (i.e., based on similarity measure only) and supervised approaches for the tasks. Finally, we present datasets for multiple choice question answering and next sentence prediction in Russian.
Tasks	Paraphrase Identification, Question Answering, Sentence Embeddings
Published	2019-10-29
URL	https://arxiv.org/abs/1910.13291v1
PDF	https://arxiv.org/pdf/1910.13291v1.pdf
PWC	https://paperswithcode.com/paper/191013291
Repo
Framework

Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning


Title	Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning
Authors	Jiangnan Xia, Chen Wu, Ming Yan
Abstract	This paper focuses on how to take advantage of external relational knowledge to improve machine reading comprehension (MRC) with multi-task learning. Most of the traditional methods in MRC assume that the knowledge used to get the correct answer generally exists in the given documents. However, in real-world task, part of knowledge may not be mentioned and machines should be equipped with the ability to leverage external knowledge. In this paper, we integrate relational knowledge into MRC model for commonsense reasoning. Specifically, based on a pre-trained language model (LM). We design two auxiliary relation-aware tasks to predict if there exists any commonsense relation and what is the relation type between two words, in order to better model the interactions between document and candidate answer option. We conduct experiments on two multi-choice benchmark datasets: the SemEval-2018 Task 11 and the Cloze Story Test. The experimental results demonstrate the effectiveness of the proposed method, which achieves superior performance compared with the comparable baselines on both datasets.
Tasks	Language Modelling, Machine Reading Comprehension, Multi-Task Learning, Reading Comprehension
Published	2019-08-13
URL	https://arxiv.org/abs/1908.04530v2
PDF	https://arxiv.org/pdf/1908.04530v2.pdf
PWC	https://paperswithcode.com/paper/incorporating-relation-knowledge-into
Repo
Framework

Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets


Title	Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets
Authors	Yogesh Balaji, Tom Goldstein, Judy Hoffman
Abstract	Adversarial training is by far the most successful strategy for improving robustness of neural networks to adversarial attacks. Despite its success as a defense mechanism, adversarial training fails to generalize well to unperturbed test set. We hypothesize that this poor generalization is a consequence of adversarial training with uniform perturbation radius around every training sample. Samples close to decision boundary can be morphed into a different class under a small perturbation budget, and enforcing large margins around these samples produce poor decision boundaries that generalize poorly. Motivated by this hypothesis, we propose instance adaptive adversarial training – a technique that enforces sample-specific perturbation margins around every training sample. We show that using our approach, test accuracy on unperturbed samples improve with a marginal drop in robustness. Extensive experiments on CIFAR-10, CIFAR-100 and Imagenet datasets demonstrate the effectiveness of our proposed approach.
Tasks
Published	2019-10-17
URL	https://arxiv.org/abs/1910.08051v1
PDF	https://arxiv.org/pdf/1910.08051v1.pdf
PWC	https://paperswithcode.com/paper/instance-adaptive-adversarial-training
Repo
Framework

Training-Free Uncertainty Estimation for Neural Networks


Title	Training-Free Uncertainty Estimation for Neural Networks
Authors	Lu Mi, Hao Wang, Yonglong Tian, Nir Shavit
Abstract	Uncertainty estimation is an essential step in the evaluation of the robustness for deep learning models in computer vision, especially when applied in risk-sensitive areas. However, most state-of-the-art deep learning models either fail to obtain uncertainty estimation or need significant modification (e.g., formulating a proper Bayesian treatment) to obtain it. None of the previous methods are able to take an arbitrary model off the shelf and generate uncertainty estimation without retraining or redesigning it. To address this gap, we perform the first systematic exploration into training-free uncertainty estimation. We propose three simple and scalable methods to analyze the variance of output from a trained network under tolerable perturbations: infer-transformation, infer-noise, and infer-dropout. They operate solely during inference, without the need to re-train, re-design, or fine-tune the model, as typically required by other state-of-the-art uncertainty estimation methods. Surprisingly, even without involving such perturbations in training, our methods produce comparable or even better uncertainty estimation when compared to other training-required state-of-the-art methods. Last but not least, we demonstrate that the uncertainty from our proposed methods can be used to improve the neural network training.
Tasks
Published	2019-09-28
URL	https://arxiv.org/abs/1910.04858v1
PDF	https://arxiv.org/pdf/1910.04858v1.pdf
PWC	https://paperswithcode.com/paper/training-free-uncertainty-estimation-for
Repo
Framework

Prognostic Value of Transfer Learning Based Features in Resectable Pancreatic Ductal Adenocarcinoma


Title	Prognostic Value of Transfer Learning Based Features in Resectable Pancreatic Ductal Adenocarcinoma
Authors	Yucheng Zhang, Edrise M. Lobo-Mueller, Paul Karanicolas, Steven Gallinger, Masoom A. Haider, Farzad Khalvati
Abstract	Pancreatic Ductal Adenocarcinoma (PDAC) is one of the most aggressive cancers with an extremely poor prognosis. Radiomics has shown prognostic ability in multiple types of cancer including PDAC. However, the prognostic value of traditional radiomics pipelines, which are based on hand-crafted radiomic features alone is limited. Convolutional neural networks (CNNs) have been shown to outperform these feature-based models in computer vision tasks. However, training a CNN from scratch needs a large sample size which is not feasible in most medical imaging studies. As an alternative solution, CNN-based transfer learning has shown potential for achieving reasonable performance using small datasets. In this work, we developed and validated a CNN-based transfer learning approach for prognostication of PDAC patients for overall survival using two independent resectable PDAC cohorts. The proposed deep transfer learning model for prognostication of PDAC achieved the area under the receiver operating characteristic curve of 0.74, which was significantly higher than that of the traditional radiomics model (0.56) as well as a CNN model trained from scratch (0.50). These results suggest that deep transfer learning may significantly improve prognosis performance using small datasets in medical imaging.
Tasks	Object Detection, Transfer Learning
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09888v2
PDF	https://arxiv.org/pdf/1905.09888v2.pdf
PWC	https://paperswithcode.com/paper/improving-prognostic-value-of-ct-deep
Repo
Framework

Higher-Dimensional Potential Heuristics for Optimal Classical Planning


Title	Higher-Dimensional Potential Heuristics for Optimal Classical Planning
Authors	Florian Pommerening, Malte Helmert, Blai Bonet
Abstract	Potential heuristics for state-space search are defined as weighted sums over simple state features. Atomic features consider the value of a single state variable in a factored state representation, while binary features consider joint assignments to two state variables. Previous work showed that the set of all admissible and consistent potential heuristics using atomic features can be characterized by a compact set of linear constraints. We generalize this result to binary features and prove a hardness result for features of higher dimension. Furthermore, we prove a tractability result based on the treewidth of a new graphical structure we call the context-dependency graph. Finally, we study the relationship of potential heuristics to transition cost partitioning. Experimental results show that binary potential heuristics are significantly more informative than the previously considered atomic ones.
Tasks
Published	2019-09-26
URL	https://arxiv.org/abs/1909.12142v1
PDF	https://arxiv.org/pdf/1909.12142v1.pdf
PWC	https://paperswithcode.com/paper/higher-dimensional-potential-heuristics-for
Repo
Framework

Captioning Near-Future Activity Sequences


Title	Captioning Near-Future Activity Sequences
Authors	Tahmida Mahmud, Mohammad Billah, Mahmudul Hasan, Amit K. Roy-Chowdhury
Abstract	Most of the existing works on human activity analysis focus on recognition or early recognition of the activity labels from complete or partial observations. Similarly, existing video captioning approaches focus on the observed events in videos. Predicting the labels and the captions of future activities where no frames of the predicted activities have been observed is a challenging problem, with important applications that require anticipatory response. In this work, we propose a system that can infer the labels and the captions of a sequence of future activities. Our proposed network for label prediction of a future activity sequence is similar to a hybrid Siamese network with three branches where the first branch takes visual features from the objects present in the scene, the second branch takes observed activity features and the third branch captures the last observed activity features. The predicted labels and the observed scene context are then mapped to meaningful captions using a sequence-to-sequence learning based method. Experiments on three challenging activity analysis datasets and a video description dataset demonstrate that both our label prediction framework and captioning framework outperforms the state-of-the-arts.
Tasks	Video Captioning, Video Description
Published	2019-08-02
URL	https://arxiv.org/abs/1908.00943v1
PDF	https://arxiv.org/pdf/1908.00943v1.pdf
PWC	https://paperswithcode.com/paper/captioning-near-future-activity-sequences
Repo
Framework

Causal Inference Under Interference And Network Uncertainty


Title	Causal Inference Under Interference And Network Uncertainty
Authors	Rohit Bhattacharya, Daniel Malinsky, Ilya Shpitser
Abstract	Classical causal and statistical inference methods typically assume the observed data consists of independent realizations. However, in many applications this assumption is inappropriate due to a network of dependences between units in the data. Methods for estimating causal effects have been developed in the setting where the structure of dependence between units is known exactly, but in practice there is often substantial uncertainty about the precise network structure. This is true, for example, in trial data drawn from vulnerable communities where social ties are difficult to query directly. In this paper we combine techniques from the structure learning and interference literatures in causal inference, proposing a general method for estimating causal effects under data dependence when the structure of this dependence is not known a priori. We demonstrate the utility of our method on synthetic datasets which exhibit network dependence.
Tasks	Causal Inference
Published	2019-06-29
URL	https://arxiv.org/abs/1907.00221v1
PDF	https://arxiv.org/pdf/1907.00221v1.pdf
PWC	https://paperswithcode.com/paper/causal-inference-under-interference-and
Repo
Framework