October 21, 2019

3392 words 16 mins read

Paper Group AWR 19

Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality. DeepAffinity: Interpretable Deep Learning of Compound-Protein Affinity through Unified Recurrent and Convolutional Neural Networks. Deep Knockoffs. Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Ne …

Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality


Title	Matching with Text Data: An Experimental Evaluation of Methods for Matching Documents and of Measuring Match Quality
Authors	Reagan Mozer, Luke Miratrix, Aaron Russell Kaufman, L. Jason Anastasopoulos
Abstract	Matching for causal inference is a well-studied problem, but standard methods fail when the units to match are text documents: the high-dimensional and rich nature of the data renders exact matching infeasible, causes propensity scores to produce incomparable matches, and makes assessing match quality difficult. In this paper, we characterize a framework for matching text documents that decomposes existing methods into: (1) the choice of text representation, and (2) the choice of distance metric. We investigate how different choices within this framework affect both the quantity and quality of matches identified through a systematic multifactor evaluation experiment using human subjects. Altogether we evaluate over 100 unique text matching methods along with 5 comparison methods taken from the literature. Our experimental results identify methods that generate matches with higher subjective match quality than current state-of-the-art techniques. We enhance the precision of these results by developing a predictive model to estimate the match quality of pairs of text documents as a function of our various distance scores. This model, which we find successfully mimics human judgment, also allows for approximate and unsupervised evaluation of new procedures. We then employ the identified best method to illustrate the utility of text matching in two applications. First, we engage with a substantive debate in the study of media bias by using text matching to control for topic selection when comparing news articles from thirteen news sources. We then show how conditioning on text data leads to more precise causal inferences in an observational study examining the effects of a medical intervention.
Tasks	Causal Inference, Text Matching
Published	2018-01-02
URL	http://arxiv.org/abs/1801.00644v7
PDF	http://arxiv.org/pdf/1801.00644v7.pdf
PWC	https://paperswithcode.com/paper/matching-with-text-data-an-experimental
Repo	https://github.com/aaronrkaufman/textmatch
Framework	none

DeepAffinity: Interpretable Deep Learning of Compound-Protein Affinity through Unified Recurrent and Convolutional Neural Networks


Title	DeepAffinity: Interpretable Deep Learning of Compound-Protein Affinity through Unified Recurrent and Convolutional Neural Networks
Authors	Mostafa Karimi, Di Wu, Zhangyang Wang, Yang Shen
Abstract	Motivation: Drug discovery demands rapid quantification of compound-protein interaction (CPI). However, there is a lack of methods that can predict compound-protein affinity from sequences alone with high applicability, accuracy, and interpretability. Results: We present a seamless integration of domain knowledges and learning-based approaches. Under novel representations of structurally-annotated protein sequences, a semi-supervised deep learning model that unifies recurrent and convolutional neural networks has been proposed to exploit both unlabeled and labeled data, for jointly encoding molecular representations and predicting affinities. Our representations and models outperform conventional options in achieving relative error in IC$_{50}$ within 5-fold for test cases and 20-fold for protein classes not included for training. Performances for new protein classes with few labeled data are further improved by transfer learning. Furthermore, separate and joint attention mechanisms are developed and embedded to our model to add to its interpretability, as illustrated in case studies for predicting and explaining selective drug-target interactions. Lastly, alternative representations using protein sequences or compound graphs and a unified RNN/GCNN-CNN model using graph CNN (GCNN) are also explored to reveal algorithmic challenges ahead. Availability: Data and source codes are available at https://github.com/Shen-Lab/DeepAffinity Supplementary Information: Supplementary data are available at http://shen-lab.github.io/deep-affinity-bioinf18-supp-rev.pdf
Tasks	Drug Discovery, Transfer Learning
Published	2018-06-20
URL	http://arxiv.org/abs/1806.07537v2
PDF	http://arxiv.org/pdf/1806.07537v2.pdf
PWC	https://paperswithcode.com/paper/deepaffinity-interpretable-deep-learning-of
Repo	https://github.com/Yindong-Zhang/GraphConvolutionDrugTargetInteration
Framework	tf

Deep Knockoffs


Title	Deep Knockoffs
Authors	Yaniv Romano, Matteo Sesia, Emmanuel J. Candès
Abstract	This paper introduces a machine for sampling approximate model-X knockoffs for arbitrary and unspecified data distributions using deep generative models. The main idea is to iteratively refine a knockoff sampling mechanism until a criterion measuring the validity of the produced knockoffs is optimized; this criterion is inspired by the popular maximum mean discrepancy in machine learning and can be thought of as measuring the distance to pairwise exchangeability between original and knockoff features. By building upon the existing model-X framework, we thus obtain a flexible and model-free statistical tool to perform controlled variable selection. Extensive numerical experiments and quantitative tests confirm the generality, effectiveness, and power of our deep knockoff machines. Finally, we apply this new method to a real study of mutations linked to changes in drug resistance in the human immunodeficiency virus.
Tasks
Published	2018-11-16
URL	http://arxiv.org/abs/1811.06687v1
PDF	http://arxiv.org/pdf/1811.06687v1.pdf
PWC	https://paperswithcode.com/paper/deep-knockoffs
Repo	https://github.com/patrickvossler18/dk_fork
Framework	pytorch

Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets


Title	Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets
Authors	Zhiming Zhou, Yuxuan Song, Lantao Yu, Hongwei Wang, Jiadong Liang, Weinan Zhang, Zhihua Zhang, Yong Yu
Abstract	In this paper, we investigate the underlying factor that leads to failure and success in the training of GANs. We study the property of the optimal discriminative function and show that in many GANs, the gradient from the optimal discriminative function is not reliable, which turns out to be the fundamental cause of failure in training of GANs. We further demonstrate that a well-defined distance metric does not necessarily guarantee the convergence of GANs. Finally, we prove in this paper that Lipschitz-continuity condition is a general solution to make the gradient of the optimal discriminative function reliable, and characterized the necessary condition where Lipschitz-continuity ensures the convergence, which leads to a broad family of valid GAN objectives under Lipschitz-continuity condition, where Wasserstein distance is one special case. We experiment with several new objectives, which are sound according to our theorems, and we found that, compared with Wasserstein distance, the outputs of the discriminator with new objectives are more stable and the final qualities of generated samples are also consistently higher than those produced by Wasserstein distance.
Tasks
Published	2018-07-02
URL	http://arxiv.org/abs/1807.00751v6
PDF	http://arxiv.org/pdf/1807.00751v6.pdf
PWC	https://paperswithcode.com/paper/understanding-the-effectiveness-of-lipschitz
Repo	https://github.com/ZhimingZhou/AM-GAN2
Framework	tf

Evaluation of Session-based Recommendation Algorithms


Title	Evaluation of Session-based Recommendation Algorithms
Authors	Malte Ludewig, Dietmar Jannach
Abstract	Recommender systems help users find relevant items of interest, for example on e-commerce or media streaming sites. Most academic research is concerned with approaches that personalize the recommendations according to long-term user profiles. In many real-world applications, however, such long-term profiles often do not exist and recommendations therefore have to be made solely based on the observed behavior of a user during an ongoing session. Given the high practical relevance of the problem, an increased interest in this problem can be observed in recent years, leading to a number of proposals for session-based recommendation algorithms that typically aim to predict the user’s immediate next actions. In this work, we present the results of an in-depth performance comparison of a number of such algorithms, using a variety of datasets and evaluation measures. Our comparison includes the most recent approaches based on recurrent neural networks like GRU4REC, factorized Markov model approaches such as FISM or FOSSIL, as well as simpler methods based, e.g., on nearest neighbor schemes. Our experiments reveal that algorithms of this latter class, despite their sometimes almost trivial nature, often perform equally well or significantly better than today’s more complex approaches based on deep neural networks. Our results therefore suggest that there is substantial room for improvement regarding the development of more sophisticated session-based recommendation algorithms.
Tasks	Recommendation Systems, Session-Based Recommendations
Published	2018-03-26
URL	http://arxiv.org/abs/1803.09587v2
PDF	http://arxiv.org/pdf/1803.09587v2.pdf
PWC	https://paperswithcode.com/paper/evaluation-of-session-based-recommendation
Repo	https://github.com/rn5l/session-rec
Framework	tf

Longitudinal data analysis using matrix completion


Title	Longitudinal data analysis using matrix completion
Authors	Łukasz Kidziński, Trevor Hastie
Abstract	In clinical practice and biomedical research, measurements are often collected sparsely and irregularly in time while the data acquisition is expensive and inconvenient. Examples include measurements of spine bone mineral density, cancer growth through mammography or biopsy, a progression of defect of vision, or assessment of gait in patients with neurological disorders. Since the data collection is often costly and inconvenient, estimation of progression from sparse observations is of great interest for practitioners. From the statistical standpoint, such data is often analyzed in the context of a mixed-effect model where time is treated as both random and fixed effect. Alternatively, researchers analyze Gaussian processes or functional data where observations are assumed to be drawn from a certain distribution of processes. These models are flexible but rely on probabilistic assumptions and require very careful implementation. In this study, we propose an alternative elementary framework for analyzing longitudinal data, relying on matrix completion. Our method yields point estimates of progression curves by iterative application of the SVD. Our framework covers multivariate longitudinal data, regression and can be easily extended to other settings. We apply our methods to understand trends of progression of motor impairment in children with Cerebral Palsy. Our model approximates individual progression curves and explains 30% of the variability. Low-rank representation of progression trends enables discovering that subtypes of Cerebral Palsy exhibit different progression trends.
Tasks	Gaussian Processes, Matrix Completion
Published	2018-09-24
URL	http://arxiv.org/abs/1809.08771v1
PDF	http://arxiv.org/pdf/1809.08771v1.pdf
PWC	https://paperswithcode.com/paper/longitudinal-data-analysis-using-matrix
Repo	https://github.com/kidzik/fcomplete
Framework	none

A Linear Constrained Optimization Benchmark For Probabilistic Search Algorithms: The Rotated Klee-Minty Problem


Title	A Linear Constrained Optimization Benchmark For Probabilistic Search Algorithms: The Rotated Klee-Minty Problem
Authors	Michael Hellwig, Hans-Georg Beyer
Abstract	The development, assessment, and comparison of randomized search algorithms heavily rely on benchmarking. Regarding the domain of constrained optimization, the number of currently available benchmark environments bears no relation to the number of distinct problem features. The present paper advances a proposal of a scalable linear constrained optimization problem that is suitable for benchmarking Evolutionary Algorithms. By comparing two recent EA variants, the linear benchmarking environment is demonstrated.
Tasks
Published	2018-07-26
URL	http://arxiv.org/abs/1807.10068v1
PDF	http://arxiv.org/pdf/1807.10068v1.pdf
PWC	https://paperswithcode.com/paper/a-linear-constrained-optimization-benchmark
Repo	https://github.com/hellwigm/RotatedKleeMintyProblem
Framework	none

Optimizing Video Object Detection via a Scale-Time Lattice


Title	Optimizing Video Object Detection via a Scale-Time Lattice
Authors	Kai Chen, Jiaqi Wang, Shuo Yang, Xingcheng Zhang, Yuanjun Xiong, Chen Change Loy, Dahua Lin
Abstract	High-performance object detection relies on expensive convolutional networks to compute features, often leading to significant challenges in applications, e.g. those that require detecting objects from video streams in real time. The key to this problem is to trade accuracy for efficiency in an effective way, i.e. reducing the computing cost while maintaining competitive performance. To seek a good balance, previous efforts usually focus on optimizing the model architectures. This paper explores an alternative approach, that is, to reallocate the computation over a scale-time space. The basic idea is to perform expensive detection sparsely and propagate the results across both scales and time with substantially cheaper networks, by exploiting the strong correlations among them. Specifically, we present a unified framework that integrates detection, temporal propagation, and across-scale refinement on a Scale-Time Lattice. On this framework, one can explore various strategies to balance performance and cost. Taking advantage of this flexibility, we further develop an adaptive scheme with the detector invoked on demand and thus obtain improved tradeoff. On ImageNet VID dataset, the proposed method can achieve a competitive mAP 79.6% at 20 fps, or 79.0% at 62 fps as a performance/speed tradeoff.
Tasks	Object Detection, Video Object Detection
Published	2018-04-16
URL	http://arxiv.org/abs/1804.05472v1
PDF	http://arxiv.org/pdf/1804.05472v1.pdf
PWC	https://paperswithcode.com/paper/optimizing-video-object-detection-via-a-scale
Repo	https://github.com/guanfuchen/video_obj
Framework	pytorch

A no-regret generalization of hierarchical softmax to extreme multi-label classification


Title	A no-regret generalization of hierarchical softmax to extreme multi-label classification
Authors	Marek Wydmuch, Kalina Jasinska, Mikhail Kuznetsov, Róbert Busa-Fekete, Krzysztof Dembczyński
Abstract	Extreme multi-label classification (XMLC) is a problem of tagging an instance with a small subset of relevant labels chosen from an extremely large pool of possible labels. Large label spaces can be efficiently handled by organizing labels as a tree, like in the hierarchical softmax (HSM) approach commonly used for multi-class problems. In this paper, we investigate probabilistic label trees (PLTs) that have been recently devised for tackling XMLC problems. We show that PLTs are a no-regret multi-label generalization of HSM when precision@k is used as a model evaluation metric. Critically, we prove that pick-one-label heuristic - a reduction technique from multi-label to multi-class that is routinely used along with HSM - is not consistent in general. We also show that our implementation of PLTs, referred to as extremeText (XT), obtains significantly better results than HSM with the pick-one-label heuristic and XML-CNN, a deep network specifically designed for XMLC problems. Moreover, XT is competitive to many state-of-the-art approaches in terms of statistical performance, model size and prediction time which makes it amenable to deploy in an online system.
Tasks	Extreme Multi-Label Classification, Multi-Label Classification
Published	2018-10-27
URL	http://arxiv.org/abs/1810.11671v1
PDF	http://arxiv.org/pdf/1810.11671v1.pdf
PWC	https://paperswithcode.com/paper/a-no-regret-generalization-of-hierarchical
Repo	https://github.com/mwydmuch/extremeText
Framework	tf

Video-to-Video Synthesis


Title	Video-to-Video Synthesis
Authors	Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, Bryan Catanzaro
Abstract	We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e.g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content of the source video. While its image counterpart, the image-to-image synthesis problem, is a popular topic, the video-to-video synthesis problem is less explored in the literature. Without understanding temporal dynamics, directly applying existing image synthesis approaches to an input video often results in temporally incoherent videos of low visual quality. In this paper, we propose a novel video-to-video synthesis approach under the generative adversarial learning framework. Through carefully-designed generator and discriminator architectures, coupled with a spatio-temporal adversarial objective, we achieve high-resolution, photorealistic, temporally coherent video results on a diverse set of input formats including segmentation masks, sketches, and poses. Experiments on multiple benchmarks show the advantage of our method compared to strong baselines. In particular, our model is capable of synthesizing 2K resolution videos of street scenes up to 30 seconds long, which significantly advances the state-of-the-art of video synthesis. Finally, we apply our approach to future video prediction, outperforming several state-of-the-art competing systems.
Tasks	Semantic Segmentation, Video Prediction, Video-to-Video Synthesis
Published	2018-08-20
URL	http://arxiv.org/abs/1808.06601v2
PDF	http://arxiv.org/pdf/1808.06601v2.pdf
PWC	https://paperswithcode.com/paper/video-to-video-synthesis
Repo	https://github.com/eric-erki/vid2vid
Framework	pytorch

ViZDoom Competitions: Playing Doom from Pixels


Title	ViZDoom Competitions: Playing Doom from Pixels
Authors	Marek Wydmuch, Michał Kempka, Wojciech Jaśkowski
Abstract	This paper presents the first two editions of Visual Doom AI Competition, held in 2016 and 2017. The challenge was to create bots that compete in a multi-player deathmatch in a first-person shooter (FPS) game, Doom. The bots had to make their decisions based solely on visual information, i.e., a raw screen buffer. To play well, the bots needed to understand their surroundings, navigate, explore, and handle the opponents at the same time. These aspects, together with the competitive multi-agent aspect of the game, make the competition a unique platform for evaluating the state of the art reinforcement learning algorithms. The paper discusses the rules, solutions, results, and statistics that give insight into the agents’ behaviors. Best-performing agents are described in more detail. The results of the competition lead to the conclusion that, although reinforcement learning can produce capable Doom bots, they still are not yet able to successfully compete against humans in this game. The paper also revisits the ViZDoom environment, which is a flexible, easy to use, and efficient 3D platform for research for vision-based reinforcement learning, based on a well-recognized first-person perspective game Doom.
Tasks
Published	2018-09-10
URL	http://arxiv.org/abs/1809.03470v1
PDF	http://arxiv.org/pdf/1809.03470v1.pdf
PWC	https://paperswithcode.com/paper/vizdoom-competitions-playing-doom-from-pixels
Repo	https://github.com/mwydmuch/ViZDoom
Framework	tf

TextField: Learning A Deep Direction Field for Irregular Scene Text Detection


Title	TextField: Learning A Deep Direction Field for Irregular Scene Text Detection
Authors	Yongchao Xu, Yukang Wang, Wei Zhou, Yongpan Wang, Zhibo Yang, Xiang Bai
Abstract	Scene text detection is an important step of scene text reading system. The main challenges lie on significantly varied sizes and aspect ratios, arbitrary orientations and shapes. Driven by recent progress in deep learning, impressive performances have been achieved for multi-oriented text detection. Yet, the performance drops dramatically in detecting curved texts due to the limited text representation (e.g., horizontal bounding boxes, rotated rectangles, or quadrilaterals). It is of great interest to detect curved texts, which are actually very common in natural scenes. In this paper, we present a novel text detector named TextField for detecting irregular scene texts. Specifically, we learn a direction field pointing away from the nearest text boundary to each text point. This direction field is represented by an image of two-dimensional vectors and learned via a fully convolutional neural network. It encodes both binary text mask and direction information used to separate adjacent text instances, which is challenging for classical segmentation-based approaches. Based on the learned direction field, we apply a simple yet effective morphological-based post-processing to achieve the final detection. Experimental results show that the proposed TextField outperforms the state-of-the-art methods by a large margin (28% and 8%) on two curved text datasets: Total-Text and CTW1500, respectively, and also achieves very competitive performance on multi-oriented datasets: ICDAR 2015 and MSRA-TD500. Furthermore, TextField is robust in generalizing to unseen datasets. The code is available at https://github.com/YukangWang/TextField.
Tasks	Scene Text Detection
Published	2018-12-04
URL	https://arxiv.org/abs/1812.01393v2
PDF	https://arxiv.org/pdf/1812.01393v2.pdf
PWC	https://paperswithcode.com/paper/textfield-learning-a-deep-direction-field-for
Repo	https://github.com/YukangWang/TextField
Framework	none

Attention Models in Graphs: A Survey


Title	Attention Models in Graphs: A Survey
Authors	John Boaz Lee, Ryan A. Rossi, Sungchul Kim, Nesreen K. Ahmed, Eunyee Koh
Abstract	Graph-structured data arise naturally in many different application domains. By representing data as graphs, we can capture entities (i.e., nodes) as well as their relationships (i.e., edges) with each other. Many useful insights can be derived from graph-structured data as demonstrated by an ever-growing body of work focused on graph mining. However, in the real-world, graphs can be both large - with many complex patterns - and noisy which can pose a problem for effective graph mining. An effective way to deal with this issue is to incorporate “attention” into graph mining solutions. An attention mechanism allows a method to focus on task-relevant parts of the graph, helping it to make better decisions. In this work, we conduct a comprehensive and focused survey of the literature on the emerging field of graph attention models. We introduce three intuitive taxonomies to group existing work. These are based on problem setting (type of input and output), the type of attention mechanism used, and the task (e.g., graph classification, link prediction, etc.). We motivate our taxonomies through detailed examples and use each to survey competing approaches from a unique standpoint. Finally, we highlight several challenges in the area and discuss promising directions for future work.
Tasks	Graph Classification, Link Prediction
Published	2018-07-20
URL	http://arxiv.org/abs/1807.07984v1
PDF	http://arxiv.org/pdf/1807.07984v1.pdf
PWC	https://paperswithcode.com/paper/attention-models-in-graphs-a-survey
Repo	https://github.com/zhliping/Deep-Learning
Framework	pytorch

On the Ineffectiveness of Variance Reduced Optimization for Deep Learning


Title	On the Ineffectiveness of Variance Reduced Optimization for Deep Learning
Authors	Aaron Defazio, Léon Bottou
Abstract	The application of stochastic variance reduction to optimization has shown remarkable recent theoretical and practical success. The applicability of these techniques to the hard non-convex optimization problems encountered during training of modern deep neural networks is an open problem. We show that naive application of the SVRG technique and related approaches fail, and explore why.
Tasks
Published	2018-12-11
URL	https://arxiv.org/abs/1812.04529v2
PDF	https://arxiv.org/pdf/1812.04529v2.pdf
PWC	https://paperswithcode.com/paper/on-the-ineffectiveness-of-variance-reduced
Repo	https://github.com/facebookresearch/deep-variance-reduction
Framework	pytorch

aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model


Title	aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model
Authors	Liu Yang, Qingyao Ai, Jiafeng Guo, W. Bruce Croft
Abstract	As an alternative to question answering methods based on feature engineering, deep learning approaches such as convolutional neural networks (CNNs) and Long Short-Term Memory Models (LSTMs) have recently been proposed for semantic matching of questions and answers. To achieve good results, however, these models have been combined with additional features such as word overlap or BM25 scores. Without this combination, these models perform significantly worse than methods based on linguistic feature engineering. In this paper, we propose an attention based neural matching model for ranking short answer text. We adopt value-shared weighting scheme instead of position-shared weighting scheme for combining different matching signals and incorporate question term importance learning using question attention network. Using the popular benchmark TREC QA data, we show that the relatively simple aNMM model can significantly outperform other neural network models that have been used for the question answering task, and is competitive with models that are combined with additional features. When aNMM is combined with additional features, it outperforms all baselines.
Tasks	Feature Engineering, Question Answering
Published	2018-01-05
URL	https://arxiv.org/abs/1801.01641v2
PDF	https://arxiv.org/pdf/1801.01641v2.pdf
PWC	https://paperswithcode.com/paper/anmm-ranking-short-answer-texts-with
Repo	https://github.com/yangliuy/aNMM-CIKM16
Framework	tf