February 1, 2020

3349 words 16 mins read

Paper Group AWR 133

Attribute-Aware Attention Model for Fine-grained Representation Learning. Prosodic Phrase Alignment for Machine Dubbing. The Brier Score under Administrative Censoring: Problems and Solutions. Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning. Continuous and Discrete-Time Survival Prediction with Neural Networks. Relation-Shape Convolu …

Attribute-Aware Attention Model for Fine-grained Representation Learning


Title	Attribute-Aware Attention Model for Fine-grained Representation Learning
Authors	Kai Han, Jianyuan Guo, Chao Zhang, Mingjian Zhu
Abstract	How to learn a discriminative fine-grained representation is a key point in many computer vision applications, such as person re-identification, fine-grained classification, fine-grained image retrieval, etc. Most of the previous methods focus on learning metrics or ensemble to derive better global representation, which are usually lack of local information. Based on the considerations above, we propose a novel Attribute-Aware Attention Model ($A^3M$), which can learn local attribute representation and global category representation simultaneously in an end-to-end manner. The proposed model contains two attention models: attribute-guided attention module uses attribute information to help select category features in different regions, at the same time, category-guided attention module selects local features of different attributes with the help of category cues. Through this attribute-category reciprocal process, local and global features benefit from each other. Finally, the resulting feature contains more intrinsic information for image recognition instead of the noisy and irrelevant features. Extensive experiments conducted on Market-1501, CompCars, CUB-200-2011 and CARS196 demonstrate the effectiveness of our $A^3M$. Code is available at https://github.com/iamhankai/attribute-aware-attention.
Tasks	Fine-Grained Image Classification, Image Retrieval, Person Re-Identification, Representation Learning
Published	2019-01-02
URL	https://arxiv.org/abs/1901.00392v2
PDF	https://arxiv.org/pdf/1901.00392v2.pdf
PWC	https://paperswithcode.com/paper/attribute-aware-attention-model-for-fine
Repo	https://github.com/iamhankai/attribute-aware-attention
Framework	none

Prosodic Phrase Alignment for Machine Dubbing


Title	Prosodic Phrase Alignment for Machine Dubbing
Authors	Alp Öktem, Mireia Farrús, Antonio Bonafonte
Abstract	Dubbing is a type of audiovisual translation where dialogues are translated and enacted so that they give the impression that the media is in the target language. It requires a careful alignment of dubbed recordings with the lip movements of performers in order to achieve visual coherence. In this paper, we deal with the specific problem of prosodic phrase synchronization within the framework of machine dubbing. Our methodology exploits the attention mechanism output in neural machine translation to find plausible phrasing for the translated dialogue lines and then uses them to condition their synthesis. Our initial work in this field records comparable speech rate ratio to professional dubbing translation, and improvement in terms of lip-syncing of long dialogue lines.
Tasks	Machine Translation
Published	2019-08-20
URL	https://arxiv.org/abs/1908.07226v1
PDF	https://arxiv.org/pdf/1908.07226v1.pdf
PWC	https://paperswithcode.com/paper/prosodic-phrase-alignment-for-machine-dubbing
Repo	https://github.com/alpoktem/MachineDub
Framework	none

The Brier Score under Administrative Censoring: Problems and Solutions


Title	The Brier Score under Administrative Censoring: Problems and Solutions
Authors	Håvard Kvamme, Ørnulf Borgan
Abstract	The Brier score is commonly used for evaluating probability predictions. In survival analysis, with right-censored observations of the event times, this score can be weighted by the inverse probability of censoring (IPCW) to retain its original interpretation. It is common practice to estimate the censoring distribution with the Kaplan-Meier estimator, even though it assumes that the censoring distribution is independent of the covariates. This paper discusses the general impact of the censoring estimates on the Brier score and shows that the estimation of the censoring distribution can be problematic. In particular, when the censoring times can be identified from the covariates, the IPCW score is no longer valid. For administratively censored data, where the potential censoring times are known for all individuals, we propose an alternative version of the Brier score. This administrative Brier score does not require estimation of the censoring distribution and is valid even if the censoring times can be identified from the covariates.
Tasks	Survival Analysis
Published	2019-12-18
URL	https://arxiv.org/abs/1912.08581v1
PDF	https://arxiv.org/pdf/1912.08581v1.pdf
PWC	https://paperswithcode.com/paper/the-brier-score-under-administrative
Repo	https://github.com/havakv/pycox
Framework	pytorch

Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning


Title	Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning
Authors	Yue Gao, Yuan Guo, Zhouhui Lian, Yingmin Tang, Jianguo Xiao
Abstract	Automatic generation of artistic glyph images is a challenging task that attracts many research interests. Previous methods either are specifically designed for shape synthesis or focus on texture transfer. In this paper, we propose a novel model, AGIS-Net, to transfer both shape and texture styles in one-stage with only a few stylized samples. To achieve this goal, we first disentangle the representations for content and style by using two encoders, ensuring the multi-content and multi-style generation. Then we utilize two collaboratively working decoders to generate the glyph shape image and its texture image simultaneously. In addition, we introduce a local texture refinement loss to further improve the quality of the synthesized textures. In this manner, our one-stage model is much more efficient and effective than other multi-stage stacked methods. We also propose a large-scale dataset with Chinese glyph images in various shape and texture styles, rendered from 35 professional-designed artistic fonts with 7,326 characters and 2,460 synthetic artistic fonts with 639 characters, to validate the effectiveness and extendability of our method. Extensive experiments on both English and Chinese artistic glyph image datasets demonstrate the superiority of our model in generating high-quality stylized glyph images against other state-of-the-art methods.
Tasks	Few-Shot Learning, Image Generation
Published	2019-10-11
URL	https://arxiv.org/abs/1910.04987v1
PDF	https://arxiv.org/pdf/1910.04987v1.pdf
PWC	https://paperswithcode.com/paper/artistic-glyph-image-synthesis-via-one-stage
Repo	https://github.com/hologerry/AGIS-Net
Framework	pytorch

Continuous and Discrete-Time Survival Prediction with Neural Networks


Title	Continuous and Discrete-Time Survival Prediction with Neural Networks
Authors	Håvard Kvamme, Ørnulf Borgan
Abstract	Application of discrete-time survival methods for continuous-time survival prediction is considered. For this purpose, a scheme for discretization of continuous-time data is proposed by considering the quantiles of the estimated event-time distribution, and, for smaller data sets, it is found to be preferable over the commonly used equidistant scheme. Furthermore, two interpolation schemes for continuous-time survival estimates are explored, both of which are shown to yield improved performance compared to the discrete-time estimates. The survival methods considered are based on the likelihood for right-censored survival data, and parameterize either the probability mass function (PMF) or the discrete-time hazard rate, both with neural networks. Through simulations and study of real-world data, the hazard rate parametrization is found to perform slightly better than the parametrization of the PMF. Inspired by these investigations, a continuous-time method is proposed by assuming that the continuous-time hazard rate is piecewise constant. The method, named PC-Hazard, is found to be highly competitive with the aforementioned methods in addition to other methods for survival prediction found in the literature.
Tasks	Survival Analysis
Published	2019-10-15
URL	https://arxiv.org/abs/1910.06724v1
PDF	https://arxiv.org/pdf/1910.06724v1.pdf
PWC	https://paperswithcode.com/paper/continuous-and-discrete-time-survival
Repo	https://github.com/havakv/pycox
Framework	pytorch

Relation-Shape Convolutional Neural Network for Point Cloud Analysis


Title	Relation-Shape Convolutional Neural Network for Point Cloud Analysis
Authors	Yongcheng Liu, Bin Fan, Shiming Xiang, Chunhong Pan
Abstract	Point cloud analysis is very challenging, as the shape implied in irregular points is difficult to capture. In this paper, we propose RS-CNN, namely, Relation-Shape Convolutional Neural Network, which extends regular grid CNN to irregular configuration for point cloud analysis. The key to RS-CNN is learning from relation, i.e., the geometric topology constraint among points. Specifically, the convolutional weight for local point set is forced to learn a high-level relation expression from predefined geometric priors, between a sampled point from this point set and the others. In this way, an inductive local representation with explicit reasoning about the spatial layout of points can be obtained, which leads to much shape awareness and robustness. With this convolution as a basic operator, RS-CNN, a hierarchical architecture can be developed to achieve contextual shape-aware learning for point cloud analysis. Extensive experiments on challenging benchmarks across three tasks verify RS-CNN achieves the state of the arts.
Tasks
Published	2019-04-16
URL	https://arxiv.org/abs/1904.07601v3
PDF	https://arxiv.org/pdf/1904.07601v3.pdf
PWC	https://paperswithcode.com/paper/relation-shape-convolutional-neural-network
Repo	https://github.com/Yochengliu/Relation-Shape-CNN
Framework	pytorch

Automatic Quality Estimation for Natural Language Generation: Ranting (Jointly Rating and Ranking)


Title	Automatic Quality Estimation for Natural Language Generation: Ranting (Jointly Rating and Ranking)
Authors	Ondřej Dušek, Karin Sevegnani, Ioannis Konstas, Verena Rieser
Abstract	We present a recurrent neural network based system for automatic quality estimation of natural language generation (NLG) outputs, which jointly learns to assign numerical ratings to individual outputs and to provide pairwise rankings of two different outputs. The latter is trained using pairwise hinge loss over scores from two copies of the rating network. We use learning to rank and synthetic data to improve the quality of ratings assigned by our system: we synthesise training pairs of distorted system outputs and train the system to rank the less distorted one higher. This leads to a 12% increase in correlation with human ratings over the previous benchmark. We also establish the state of the art on the dataset of relative rankings from the E2E NLG Challenge (Du\v{s}ek et al., 2019), where synthetic data lead to a 4% accuracy increase over the base model.
Tasks	Learning-To-Rank, Text Generation
Published	2019-10-10
URL	https://arxiv.org/abs/1910.04731v1
PDF	https://arxiv.org/pdf/1910.04731v1.pdf
PWC	https://paperswithcode.com/paper/automatic-quality-estimation-for-natural
Repo	https://github.com/tuetschek/ratpred
Framework	none

Graph-based Semi-Supervised & Active Learning for Edge Flows


Title	Graph-based Semi-Supervised & Active Learning for Edge Flows
Authors	Junteng Jia, Michael T. Schaub, Santiago Segarra, Austin R. Benson
Abstract	We present a graph-based semi-supervised learning (SSL) method for learning edge flows defined on a graph. Specifically, given flow measurements on a subset of edges, we want to predict the flows on the remaining edges. To this end, we develop a computational framework that imposes certain constraints on the overall flows, such as (approximate) flow conservation. These constraints render our approach different from classical graph-based SSL for vertex labels, which posits that tightly connected nodes share similar labels and leverages the graph structure accordingly to extrapolate from a few vertex labels to the unlabeled vertices. We derive bounds for our method’s reconstruction error and demonstrate its strong performance on synthetic and real-world flow networks from transportation, physical infrastructure, and the Web. Furthermore, we provide two active learning algorithms for selecting informative edges on which to measure flow, which has applications for optimal sensor deployment. The first strategy selects edges to minimize the reconstruction error bound and works well on flows that are approximately divergence-free. The second approach clusters the graph and selects bottleneck edges that cross cluster-boundaries, which works well on flows with global trends.
Tasks	Active Learning
Published	2019-05-17
URL	https://arxiv.org/abs/1905.07451v1
PDF	https://arxiv.org/pdf/1905.07451v1.pdf
PWC	https://paperswithcode.com/paper/graph-based-semi-supervised-active-learning
Repo	https://github.com/000Justin000/ssl_edge
Framework	none

Reward Learning for Efficient Reinforcement Learning in Extractive Document Summarisation


Title	Reward Learning for Efficient Reinforcement Learning in Extractive Document Summarisation
Authors	Yang Gao, Christian M. Meyer, Mohsen Mesgar, Iryna Gurevych
Abstract	Document summarisation can be formulated as a sequential decision-making problem, which can be solved by Reinforcement Learning (RL) algorithms. The predominant RL paradigm for summarisation learns a cross-input policy, which requires considerable time, data and parameter tuning due to the huge search spaces and the delayed rewards. Learning input-specific RL policies is a more efficient alternative but so far depends on handcrafted rewards, which are difficult to design and yield poor performance. We propose RELIS, a novel RL paradigm that learns a reward function with Learning-to-Rank (L2R) algorithms at training time and uses this reward function to train an input-specific RL policy at test time. We prove that RELIS guarantees to generate near-optimal summaries with appropriate L2R and RL algorithms. Empirically, we evaluate our approach on extractive multi-document summarisation. We show that RELIS reduces the training time by two orders of magnitude compared to the state-of-the-art models while performing on par with them.
Tasks	Decision Making, Learning-To-Rank
Published	2019-07-30
URL	https://arxiv.org/abs/1907.12894v1
PDF	https://arxiv.org/pdf/1907.12894v1.pdf
PWC	https://paperswithcode.com/paper/reward-learning-for-efficient-reinforcement
Repo	https://github.com/UKPLab/ijcai2019-relis
Framework	none

Relational Pooling for Graph Representations


Title	Relational Pooling for Graph Representations
Authors	Ryan L. Murphy, Balasubramaniam Srinivasan, Vinayak Rao, Bruno Ribeiro
Abstract	This work generalizes graph neural networks (GNNs) beyond those based on the Weisfeiler-Lehman (WL) algorithm, graph Laplacians, and diffusions. Our approach, denoted Relational Pooling (RP), draws from the theory of finite partial exchangeability to provide a framework with maximal representation power for graphs. RP can work with existing graph representation models and, somewhat counterintuitively, can make them even more powerful than the original WL isomorphism test. Additionally, RP allows architectures like Recurrent Neural Networks and Convolutional Neural Networks to be used in a theoretically sound approach for graph classification. We demonstrate improved performance of RP-based graph representations over state-of-the-art methods on a number of tasks.
Tasks	Graph Classification
Published	2019-03-06
URL	https://arxiv.org/abs/1903.02541v2
PDF	https://arxiv.org/pdf/1903.02541v2.pdf
PWC	https://paperswithcode.com/paper/relational-pooling-for-graph-representations
Repo	https://github.com/PurdueMINDS/RelationalPooling
Framework	pytorch


Title	Fake News Detection on Social Media using Geometric Deep Learning
Authors	Federico Monti, Fabrizio Frasca, Davide Eynard, Damon Mannion, Michael M. Bronstein
Abstract	Social media are nowadays one of the main news sources for millions of people around the globe due to their low cost, easy access and rapid dissemination. This however comes at the cost of dubious trustworthiness and significant risk of exposure to ‘fake news’, intentionally written to mislead the readers. Automatically detecting fake news poses challenges that defy existing content-based analysis approaches. One of the main reasons is that often the interpretation of the news requires the knowledge of political or social context or ‘common sense’, which current NLP algorithms are still missing. Recent studies have shown that fake and real news spread differently on social media, forming propagation patterns that could be harnessed for the automatic fake news detection. Propagation-based approaches have multiple advantages compared to their content-based counterparts, among which is language independence and better resilience to adversarial attacks. In this paper we show a novel automatic fake news detection model based on geometric deep learning. The underlying core algorithms are a generalization of classical CNNs to graphs, allowing the fusion of heterogeneous data such as content, user profile and activity, social graph, and news propagation. Our model was trained and tested on news stories, verified by professional fact-checking organizations, that were spread on Twitter. Our experiments indicate that social network structure and propagation are important features allowing highly accurate (92.7% ROC AUC) fake news detection. Second, we observe that fake news can be reliably detected at an early stage, after just a few hours of propagation. Third, we test the aging of our model on training and testing data separated in time. Our results point to the promise of propagation-based approaches for fake news detection as an alternative or complementary strategy to content-based approaches.
Tasks	Common Sense Reasoning, Fake News Detection, Graph Classification
Published	2019-02-10
URL	http://arxiv.org/abs/1902.06673v1
PDF	http://arxiv.org/pdf/1902.06673v1.pdf
PWC	https://paperswithcode.com/paper/fake-news-detection-on-social-media-using
Repo	https://github.com/kc-ml2/ipam-2019-dgl
Framework	pytorch

DAS3H: Modeling Student Learning and Forgetting for Optimally Scheduling Distributed Practice of Skills


Title	DAS3H: Modeling Student Learning and Forgetting for Optimally Scheduling Distributed Practice of Skills
Authors	Benoît Choffin, Fabrice Popineau, Yolaine Bourda, Jill-Jênn Vie
Abstract	Spaced repetition is among the most studied learning strategies in the cognitive science literature. It consists in temporally distributing exposure to an information so as to improve long-term memorization. Providing students with an adaptive and personalized distributed practice schedule would benefit more than just a generic scheduler. However, the applicability of such adaptive schedulers seems to be limited to pure memorization, e.g. flashcards or foreign language learning. In this article, we first frame the research problem of optimizing an adaptive and personalized spaced repetition scheduler when memorization concerns the application of underlying multiple skills. To this end, we choose to rely on a student model for inferring knowledge state and memory dynamics on any skill or combination of skills. We argue that no knowledge tracing model takes both memory decay and multiple skill tagging into account for predicting student performance. As a consequence, we propose a new student learning and forgetting model suited to our research problem: DAS3H builds on the additive factor models and includes a representation of the temporal distribution of past practice on the skills involved by an item. In particular, DAS3H allows the learning and forgetting curves to differ from one skill to another. Finally, we provide empirical evidence on three real-world educational datasets that DAS3H outperforms other state-of-the-art EDM models. These results suggest that incorporating both item-skill relationships and forgetting effect improves over student models that consider one or the other.
Tasks	Knowledge Tracing
Published	2019-05-14
URL	https://arxiv.org/abs/1905.06873v1
PDF	https://arxiv.org/pdf/1905.06873v1.pdf
PWC	https://paperswithcode.com/paper/das3h-modeling-student-learning-and
Repo	https://github.com/BenoitChoffin/das3h
Framework	none

Variational Recurrent Neural Networks for Graph Classification


Title	Variational Recurrent Neural Networks for Graph Classification
Authors	Edouard Pineau, Nathan de Lara
Abstract	We address the problem of graph classification based only on structural information. Inspired by natural language processing techniques (NLP), our model sequentially embeds information to estimate class membership probabilities. Besides, we experiment with NLP-like variational regularization techniques, making the model predict the next node in the sequence as it reads it. We experimentally show that our model achieves state-of-the-art classification results on several standard molecular datasets. Finally, we perform a qualitative analysis and give some insights on whether the node prediction helps the model better classify graphs.
Tasks	Graph Classification
Published	2019-02-07
URL	https://arxiv.org/abs/1902.02721v4
PDF	https://arxiv.org/pdf/1902.02721v4.pdf
PWC	https://paperswithcode.com/paper/graph-classification-with-recurrent
Repo	https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification
Framework	pytorch

A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning


Title	A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning
Authors	Shahbaz Rezaei, Xin Liu
Abstract	Due to insufficient training data and the high computational cost to train a deep neural network from scratch, transfer learning has been extensively used in many deep-neural-network-based applications. A commonly used transfer learning approach involves taking a part of a pre-trained model, adding a few layers at the end, and re-training the new layers with a small dataset. This approach, while efficient and widely used, imposes a security vulnerability because the pre-trained model used in transfer learning is usually publicly available, including to potential attackers. In this paper, we show that without any additional knowledge other than the pre-trained model, an attacker can launch an effective and efficient brute force attack that can craft instances of input to trigger each target class with high confidence. We assume that the attacker has no access to any target-specific information, including samples from target classes, re-trained model, and probabilities assigned by Softmax to each class, and thus making the attack target-agnostic. These assumptions render all previous attack models inapplicable, to the best of our knowledge. To evaluate the proposed attack, we perform a set of experiments on face recognition and speech recognition tasks and show the effectiveness of the attack. Our work reveals a fundamental security weakness of the Softmax layer when used in transfer learning settings.
Tasks	Face Recognition, Image Classification, Speech Recognition, Transfer Learning
Published	2019-04-08
URL	https://arxiv.org/abs/1904.04334v3
PDF	https://arxiv.org/pdf/1904.04334v3.pdf
PWC	https://paperswithcode.com/paper/a-target-agnostic-attack-on-deep-models
Repo	https://github.com/shrezaei/Target-Agnostic-Attack
Framework	none

EdNet: A Large-Scale Hierarchical Dataset in Education


Title	EdNet: A Large-Scale Hierarchical Dataset in Education
Authors	Youngduck Choi, Youngnam Lee, Dongmin Shin, Junghyun Cho, Seoyon Park, Seewoo Lee, Jineon Baek, Byungsoo Kim, Youngjun Jang
Abstract	With advances in Artificial Intelligence in Education (AIEd) and the ever-growing scale of Interactive Educational Systems (IESs), data-driven approach has become a common recipe for various tasks such as knowledge tracing and learning path recommendation. Unfortunately, collecting real students’ interaction data is often challenging, which results in the lack of public large-scale benchmark dataset reflecting a wide variety of student behaviors in modern IESs. Although several datasets, such as ASSISTments, Junyi Academy, Synthetic and STATICS, are publicly available and widely used, they are not large enough to leverage the full potential of state-of-the-art data-driven models and limits the recorded behaviors to question-solving activities. To this end, we introduce EdNet, a large-scale hierarchical dataset of diverse student activities collected by Santa, a multi-platform self-study solution equipped with artificial intelligence tutoring system. EdNet contains 131,441,538 interactions from 784,309 students collected over more than 2 years, which is the largest among the ITS datasets released to the public so far. Unlike existing datasets, EdNet provides a wide variety of student actions ranging from question-solving to lecture consumption and item purchasing. Also, EdNet has a hierarchical structure where the student actions are divided into 4 different levels of abstractions. The features of EdNet are domain-agnostic, allowing EdNet to be extended to different domains easily. The dataset is publicly released under Creative Commons Attribution-NonCommercial 4.0 International license for research purposes. We plan to host challenges in multiple AIEd tasks with EdNet to provide a common ground for the fair comparison between different state of the art models and encourage the development of practical and effective methods.
Tasks	Knowledge Tracing
Published	2019-12-06
URL	https://arxiv.org/abs/1912.03072v1
PDF	https://arxiv.org/pdf/1912.03072v1.pdf
PWC	https://paperswithcode.com/paper/ednet-a-large-scale-hierarchical-dataset-in
Repo	https://github.com/riiid/ednet
Framework	none