April 3, 2020

2669 words 13 mins read

Paper Group AWR 45

Towards Supervised and Unsupervised Neural Machine Translation Baselines for Nigerian Pidgin. Open Knowledge Enrichment for Long-tail Entities. Decision Trees for Decision-Making under the Predict-then-Optimize Framework. Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods. Adversarial Attacks a …

Towards Supervised and Unsupervised Neural Machine Translation Baselines for Nigerian Pidgin


Title	Towards Supervised and Unsupervised Neural Machine Translation Baselines for Nigerian Pidgin
Authors	Orevaoghene Ahia, Kelechi Ogueji
Abstract	Nigerian Pidgin is arguably the most widely spoken language in Nigeria. Variants of this language are also spoken across West and Central Africa, making it a very important language. This work aims to establish supervised and unsupervised neural machine translation (NMT) baselines between English and Nigerian Pidgin. We implement and compare NMT models with different tokenization methods, creating a solid foundation for future works.
Tasks	Machine Translation, Tokenization
Published	2020-03-27
URL	https://arxiv.org/abs/2003.12660v1
PDF	https://arxiv.org/pdf/2003.12660v1.pdf
PWC	https://paperswithcode.com/paper/towards-supervised-and-unsupervised-neural
Repo	https://github.com/orevaoghene/pidgin-baseline
Framework	none

Open Knowledge Enrichment for Long-tail Entities


Title	Open Knowledge Enrichment for Long-tail Entities
Authors	Ermei Cao, Difeng Wang, Jiacheng Huang, Wei Hu
Abstract	Knowledge bases (KBs) have gradually become a valuable asset for many AI applications. While many current KBs are quite large, they are widely acknowledged as incomplete, especially lacking facts of long-tail entities, e.g., less famous persons. Existing approaches enrich KBs mainly on completing missing links or filling missing values. However, they only tackle a part of the enrichment problem and lack specific considerations regarding long-tail entities. In this paper, we propose a full-fledged approach to knowledge enrichment, which predicts missing properties and infers true facts of long-tail entities from the open Web. Prior knowledge from popular entities is leveraged to improve every enrichment step. Our experiments on the synthetic and real-world datasets and comparison with related work demonstrate the feasibility and superiority of the approach.
Tasks
Published	2020-02-15
URL	https://arxiv.org/abs/2002.06397v2
PDF	https://arxiv.org/pdf/2002.06397v2.pdf
PWC	https://paperswithcode.com/paper/open-knowledge-enrichment-for-long-tail
Repo	https://github.com/nju-websoft/OKELE
Framework	tf

Decision Trees for Decision-Making under the Predict-then-Optimize Framework


Title	Decision Trees for Decision-Making under the Predict-then-Optimize Framework
Authors	Adam N. Elmachtoub, Jason Cheuk Nam Liang, Ryan McNellis
Abstract	We consider the use of decision trees for decision-making problems under the predict-then-optimize framework. That is, we would like to first use a decision tree to predict unknown input parameters of an optimization problem, and then make decisions by solving the optimization problem using the predicted parameters. A natural loss function in this framework is to measure the suboptimality of the decisions induced by the predicted input parameters, as opposed to measuring loss using input parameter prediction error. This natural loss function is known in the literature as the Smart Predict-then-Optimize (SPO) loss, and we propose a tractable methodology called SPO Trees (SPOTs) for training decision trees under this loss. SPOTs benefit from the interpretability of decision trees, providing an interpretable segmentation of contextual features into groups with distinct optimal solutions to the optimization problem of interest. We conduct several numerical experiments on synthetic and real data including the prediction of travel times for shortest path problems and predicting click probabilities for news article recommendation. We demonstrate on these datasets that SPOTs simultaneously provide higher quality decisions and significantly lower model complexity than other machine learning approaches (e.g., CART) trained to minimize prediction error.
Tasks	Decision Making
Published	2020-02-29
URL	https://arxiv.org/abs/2003.00360v1
PDF	https://arxiv.org/pdf/2003.00360v1.pdf
PWC	https://paperswithcode.com/paper/decision-trees-for-decision-making-under-the
Repo	https://github.com/rtm2130/SPOTree
Framework	none

Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods


Title	Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods
Authors	Maria Ryskina, Ella Rabinovich, Taylor Berg-Kirkpatrick, David R. Mortensen, Yulia Tsvetkov
Abstract	We perform statistical analysis of the phenomenon of neology, the process by which new words emerge in a language, using large diachronic corpora of English. We investigate the importance of two factors, semantic sparsity and frequency growth rates of semantic neighbors, formalized in the distributional semantics paradigm. We show that both factors are predictive of word emergence although we find more support for the latter hypothesis. Besides presenting a new linguistic application of distributional semantics, this study tackles the linguistic question of the role of language-internal factors (in our case, sparsity) in language change motivated by language-external factors (reflected in frequency growth).
Tasks
Published	2020-01-21
URL	https://arxiv.org/abs/2001.07740v1
PDF	https://arxiv.org/pdf/2001.07740v1.pdf
PWC	https://paperswithcode.com/paper/where-new-words-are-born-distributional
Repo	https://github.com/ryskina/neology
Framework	none

Adversarial Attacks and Defenses on Graphs: A Review and Empirical Study


Title	Adversarial Attacks and Defenses on Graphs: A Review and Empirical Study
Authors	Wei Jin, Yaxin Li, Han Xu, Yiqi Wang, Jiliang Tang
Abstract	Deep neural networks (DNNs) have achieved significant performance in various tasks. However, recent studies have shown that DNNs can be easily fooled by small perturbation on the input, called adversarial attacks. As the extensions of DNNs to graphs, Graph Neural Networks (GNNs) have been demonstrated to inherit this vulnerability. Adversary can mislead GNNs to give wrong predictions by modifying the graph structure such as manipulating a few edges. This vulnerability has arisen tremendous concerns for adapting GNNs in safety-critical applications and has attracted increasing research attention in recent years. Thus, it is necessary and timely to provide a comprehensive overview of existing graph adversarial attacks and the countermeasures. In this survey, we categorize existing attacks and defenses, and review the corresponding state-of-the-art methods. Furthermore, we have developed a repository with representative algorithms (https://github.com/DSE-MSU/DeepRobust/tree/master/deeprobust/graph). The repository enables us to conduct empirical studies to deepen our understandings on attacks and defenses on graphs.
Tasks
Published	2020-03-02
URL	https://arxiv.org/abs/2003.00653v2
PDF	https://arxiv.org/pdf/2003.00653v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-attacks-and-defenses-on-graphs-a
Repo	https://github.com/DSE-MSU/DeepRobust
Framework	pytorch

SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models


Title	SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models
Authors	Bin Wang, C. -C. Jay Kuo
Abstract	Sentence embedding is an important research topic in natural language processing (NLP) since it can transfer knowledge to downstream tasks. Meanwhile, a contextualized word representation, called BERT, achieves the state-of-the-art performance in quite a few NLP tasks. Yet, it is an open problem to generate a high quality sentence representation from BERT-based word models. It was shown in previous study that different layers of BERT capture different linguistic properties. This allows us to fusion information across layers to find better sentence representation. In this work, we study the layer-wise pattern of the word representation of deep contextualized models. Then, we propose a new sentence embedding method by dissecting BERT-based word models through geometric analysis of the space spanned by the word representation. It is called the SBERT-WK method. No further training is required in SBERT-WK. We evaluate SBERT-WK on semantic textual similarity and downstream supervised tasks. Furthermore, ten sentence-level probing tasks are presented for detailed linguistic analysis. Experiments show that SBERT-WK achieves the state-of-the-art performance. Our codes are publicly available.
Tasks	Semantic Textual Similarity, Sentence Embedding
Published	2020-02-16
URL	https://arxiv.org/abs/2002.06652v1
PDF	https://arxiv.org/pdf/2002.06652v1.pdf
PWC	https://paperswithcode.com/paper/sbert-wk-a-sentence-embedding-method-by
Repo	https://github.com/BinWang28/SBERT-WK-Sentence-Embedding
Framework	pytorch

Learning to Shade Hand-drawn Sketches


Title	Learning to Shade Hand-drawn Sketches
Authors	Qingyuan Zheng, Zhuoru Li, Adam Bargteil
Abstract	We present a fully automatic method to generate detailed and accurate artistic shadows from pairs of line drawing sketches and lighting directions. We also contribute a new dataset of one thousand examples of pairs of line drawings and shadows that are tagged with lighting directions. Remarkably, the generated shadows quickly communicate the underlying 3D structure of the sketched scene. Consequently, the shadows generated by our approach can be used directly or as an excellent starting point for artists. We demonstrate that the deep learning network we propose takes a hand-drawn sketch, builds a 3D model in latent space, and renders the resulting shadows. The generated shadows respect the hand-drawn lines and underlying 3D space and contain sophisticated and accurate details, such as self-shadowing effects. Moreover, the generated shadows contain artistic effects, such as rim lighting or halos appearing from back lighting, that would be achievable with traditional 3D rendering methods.
Tasks
Published	2020-02-26
URL	https://arxiv.org/abs/2002.11812v1
PDF	https://arxiv.org/pdf/2002.11812v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-shade-hand-drawn-sketches
Repo	https://github.com/qyzdao/ShadeSketch
Framework	tf

Solving Satisfiability of Polynomial Formulas By Sample-Cell Projection


Title	Solving Satisfiability of Polynomial Formulas By Sample-Cell Projection
Authors	Haokun Li, Bican Xia
Abstract	A new algorithm for deciding the satisfiability of polynomial formulas over the reals is proposed. The key point of the algorithm is a new projection operator, called sample-cell projection operator, custom-made for Conflict-Driven Clause Learning (CDCL)-style search. Although the new operator is also a CAD (Cylindrical Algebraic Decomposition)-like projection operator which computes the cell (not necessarily cylindrical) containing a given sample such that each polynomial from the problem is sign-invariant on the cell, it is of singly exponential time complexity. The sample-cell projection operator can efficiently guide CDCL-style search away from conflicting states. Experiments show the effectiveness of the new algorithm.
Tasks
Published	2020-03-01
URL	https://arxiv.org/abs/2003.00409v2
PDF	https://arxiv.org/pdf/2003.00409v2.pdf
PWC	https://paperswithcode.com/paper/solving-satisfiability-of-polynomial-formulas
Repo	https://github.com/lihaokun/LiMbS
Framework	none

Improving the Evaluation of Generative Models with Fuzzy Logic


Title	Improving the Evaluation of Generative Models with Fuzzy Logic
Authors	Julian Niedermeier, Gonçalo Mordido, Christoph Meinel
Abstract	Objective and interpretable metrics to evaluate current artificial intelligent systems are of great importance, not only to analyze the current state of such systems but also to objectively measure progress in the future. In this work, we focus on the evaluation of image generation tasks. We propose a novel approach, called Fuzzy Topology Impact (FTI), that determines both the quality and diversity of an image set using topology representations combined with fuzzy logic. When compared to current evaluation methods, FTI shows better and more stable performance on multiple experiments evaluating the sensitivity to noise, mode dropping and mode inventing.
Tasks	Image Generation
Published	2020-02-03
URL	https://arxiv.org/abs/2002.03772v1
PDF	https://arxiv.org/pdf/2002.03772v1.pdf
PWC	https://paperswithcode.com/paper/improving-the-evaluation-of-generative-models
Repo	https://github.com/sleighsoft/fti
Framework	tf

Natural Image Matting via Guided Contextual Attention


Title	Natural Image Matting via Guided Contextual Attention
Authors	Yaoyi Li, Hongtao Lu
Abstract	Over the last few years, deep learning based approaches have achieved outstanding improvements in natural image matting. Many of these methods can generate visually plausible alpha estimations, but typically yield blurry structures or textures in the semitransparent area. This is due to the local ambiguity of transparent objects. One possible solution is to leverage the far-surrounding information to estimate the local opacity. Traditional affinity-based methods often suffer from the high computational complexity, which are not suitable for high resolution alpha estimation. Inspired by affinity-based method and the successes of contextual attention in inpainting, we develop a novel end-to-end approach for natural image matting with a guided contextual attention module, which is specifically designed for image matting. Guided contextual attention module directly propagates high-level opacity information globally based on the learned low-level affinity. The proposed method can mimic information flow of affinity-based methods and utilize rich features learned by deep neural networks simultaneously. Experiment results on Composition-1k testing set and alphamatting.com benchmark dataset demonstrate that our method outperforms state-of-the-art approaches in natural image matting. Code and models are available at https://github.com/Yaoyi-Li/GCA-Matting.
Tasks	Image Matting
Published	2020-01-13
URL	https://arxiv.org/abs/2001.04069v1
PDF	https://arxiv.org/pdf/2001.04069v1.pdf
PWC	https://paperswithcode.com/paper/natural-image-matting-via-guided-contextual
Repo	https://github.com/Yaoyi-Li/GCA-Matting
Framework	pytorch

Schema2QA: Answering Complex Queries on the Structured Web with a Neural Model


Title	Schema2QA: Answering Complex Queries on the Structured Web with a Neural Model
Authors	Silei Xu, Giovanni Campagna, Jian Li, Monica S. Lam
Abstract	Virtual assistants have proprietary third-party skill platforms; they train and own the voice interface to websites based on their submitted skill information. This paper proposes Schema2QA, an open-source toolkit that leverages the Schema.org markup found in many websites to automatically build skills. Schema2QA has several advantages: (1) Schema2QA is more accurate than commercial assistants in answering compositional queries involving multiple properties; (2) it has a low-cost training data acquisition methodology that requires only writing a small number of annotations per domain and paraphrasing a small number of sentences. Schema2QA uses a novel neural model that combines the BERT pretrained model with an LSTM; the model can generalize a training set containing only synthesized and paraphrase data to understand real-world crowdsourced questions. We apply Schema2QA to two different domains, showing that the skills we built can answer useful queries with little manual effort. Our skills achieve an overall accuracy between 73% and 81%. With transfer learning, we show that a new domain can achieve a 59% accuracy without manual effort. The open-source Schema2QA lets each website create and own its linguistic interface.
Tasks	Transfer Learning
Published	2020-01-16
URL	https://arxiv.org/abs/2001.05609v2
PDF	https://arxiv.org/pdf/2001.05609v2.pdf
PWC	https://paperswithcode.com/paper/schema2qa-answering-complex-queries-on-the
Repo	https://github.com/stanford-oval/genienlp
Framework	pytorch

Layered Embeddings for Amodal Instance Segmentation


Title	Layered Embeddings for Amodal Instance Segmentation
Authors	Yanfeng Liu, Eric Psota, Lance Pérez
Abstract	The proposed method extends upon the representational output of semantic instance segmentation by explicitly including both visible and occluded parts. A fully convolutional network is trained to produce consistent pixel-level embedding across two layers such that, when clustered, the results convey the full spatial extent and depth ordering of each instance. Results demonstrate that the network can accurately estimate complete masks in the presence of occlusion and outperform leading top-down bounding-box approaches. Source code available at https://github.com/yanfengliu/layered_embeddings
Tasks	Instance Segmentation, Semantic Segmentation
Published	2020-02-14
URL	https://arxiv.org/abs/2002.06264v1
PDF	https://arxiv.org/pdf/2002.06264v1.pdf
PWC	https://paperswithcode.com/paper/layered-embeddings-for-amodal-instance
Repo	https://github.com/yanfengliu/layered_embeddings
Framework	none

PyMatting: A Python Library for Alpha Matting


Title	PyMatting: A Python Library for Alpha Matting
Authors	Thomas Germer, Tobias Uelwer, Stefan Conrad, Stefan Harmeling
Abstract	An important step of many image editing tasks is to extract specific objects from an image in order to place them in a scene of a movie or compose them onto another background. Alpha matting describes the problem of separating the objects in the foreground from the background of an image given only a rough sketch. We introduce the PyMatting package for Python which implements various approaches to solve the alpha matting problem. Our toolbox is also able to extract the foreground of an image given the alpha matte. The implementation aims to be computationally efficient and easy to use. The source code of PyMatting is available under an open-source license at https://github.com/pymatting/pymatting.
Tasks
Published	2020-03-25
URL	https://arxiv.org/abs/2003.12382v1
PDF	https://arxiv.org/pdf/2003.12382v1.pdf
PWC	https://paperswithcode.com/paper/pymatting-a-python-library-for-alpha-matting
Repo	https://github.com/pymatting/pymatting
Framework	none

Authorship Attribution in Bangla literature using Character-level CNN


Title	Authorship Attribution in Bangla literature using Character-level CNN
Authors	Aisha Khatun, Anisur Rahman, Md. Saiful Islam, Marium-E-Jannat
Abstract	Characters are the smallest unit of text that can extract stylometric signals to determine the author of a text. In this paper, we investigate the effectiveness of character-level signals in Authorship Attribution of Bangla Literature and show that the results are promising but improvable. The time and memory efficiency of the proposed model is much higher than the word level counterparts but accuracy is 2-5% less than the best performing word-level models. Comparison of various word-based models is performed and shown that the proposed model performs increasingly better with larger datasets. We also analyze the effect of pre-training character embedding of diverse Bangla character set in authorship attribution. It is seen that the performance is improved by up to 10% on pre-training. We used 2 datasets from 6 to 14 authors, balancing them before training and compare the results.
Tasks
Published	2020-01-11
URL	https://arxiv.org/abs/2001.05316v1
PDF	https://arxiv.org/pdf/2001.05316v1.pdf
PWC	https://paperswithcode.com/paper/authorship-attribution-in-bangla-literature
Repo	https://github.com/anutkk/RambaNet
Framework	tf

Neural Machine Translation with Joint Representation


Title	Neural Machine Translation with Joint Representation
Authors	Yanyang Li, Qiang Wang, Tong Xiao, Tongran Liu, Jingbo Zhu
Abstract	Though early successes of Statistical Machine Translation (SMT) systems are attributed in part to the explicit modelling of the interaction between any two source and target units, e.g., alignment, the recent Neural Machine Translation (NMT) systems resort to the attention which partially encodes the interaction for efficiency. In this paper, we employ Joint Representation that fully accounts for each possible interaction. We sidestep the inefficiency issue by refining representations with the proposed efficient attention operation. The resulting Reformer models offer a new Sequence-to- Sequence modelling paradigm besides the Encoder-Decoder framework and outperform the Transformer baseline in either the small scale IWSLT14 German-English, English-German and IWSLT15 Vietnamese-English or the large scale NIST12 Chinese-English translation tasks by about 1 BLEU point.We also propose a systematic model scaling approach, allowing the Reformer model to beat the state-of-the-art Transformer in IWSLT14 German-English and NIST12 Chinese-English with about 50% fewer parameters. The code is publicly available at https://github.com/lyy1994/reformer.
Tasks	Machine Translation
Published	2020-02-16
URL	https://arxiv.org/abs/2002.06546v2
PDF	https://arxiv.org/pdf/2002.06546v2.pdf
PWC	https://paperswithcode.com/paper/neural-machine-translation-with-joint
Repo	https://github.com/lyy1994/reformer
Framework	pytorch