Paper Group NANR 88
GEOMETRIC AUGMENTATION FOR ROBUST NEURAL NETWORK CLASSIFIERS. Knowledge Graph Embedding via Graph Attenuated Attention Networks. Inverse Procedural Modeling of Knitwear. DISTRIBUTIONAL CONCAVITY REGULARIZATION FOR GANS. CDeepEx: Contrastive Deep Explanations. Dynamically Composing Domain-Data Selection with Clean-Data Selection by ``Co-Curricular L …
GEOMETRIC AUGMENTATION FOR ROBUST NEURAL NETWORK CLASSIFIERS
| Title | GEOMETRIC AUGMENTATION FOR ROBUST NEURAL NETWORK CLASSIFIERS |
| Authors | Robert M. Taylor, Yusong Tan |
| Abstract | We introduce a novel geometric perspective and unsupervised model augmentation framework for transforming traditional deep (convolutional) neural networks into adversarially robust classifiers. Class-conditional probability densities based on Bayesian nonparametric mixtures of factor analyzers (BNP-MFA) over the input space are used to design soft decision labels for feature to label isometry. Classconditional distributions over features are also learned using BNP-MFA to develop plug-in maximum a posterior (MAP) classifiers to replace the traditional multinomial logistic softmax classification layers. This novel unsupervised augmented framework, which we call geometrically robust networks (GRN), is applied to CIFAR-10, CIFAR-100, and to Radio-ML (a time series dataset for radio modulation recognition). We demonstrate the robustness of GRN models to adversarial attacks from fast gradient sign method, Carlini-Wagner, and projected gradient descent. |
| Tasks | Time Series |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=BJeapjA5FX |
| https://openreview.net/pdf?id=BJeapjA5FX | |
| PWC | https://paperswithcode.com/paper/geometric-augmentation-for-robust-neural |
| Repo | |
| Framework | |
Knowledge Graph Embedding via Graph Attenuated Attention Networks
| Title | Knowledge Graph Embedding via Graph Attenuated Attention Networks |
| Authors | Rui Wang, Bicheng Li, Shengwei Hu, Wenqian Du, Min Zhang |
| Abstract | Knowledge graphs contain a wealth of real-world knowledge that can provide strong support for artificial intelligence applications. Much progress has been made in knowledge graph completion, state-of-the-art models are based on graph convolutional neural networks. These models automatically extract features, in combination with the features of the graph model, to generate feature embeddings with a strong expressive ability. However, these methods assign the same weights on the relation path in the knowledge graph and ignore the rich information presented in neighbor nodes, which result in incomplete mining of triple features. To this end, we propose Graph Attenuated Attention networks(GAATs), a novel representation method, which integrates an attenuated attention mechanism to assign different weight in different relation path and acquire the information from the neighborhoods. As a result, entities and relations can be learned in any neighbors. Our empirical research provides insight into the effectiveness of the attenuated attention-based models, and we show significant improvement compared to the state-of-the-art methods on two benchmark datasets WN18RR and FB15k-237. |
| Tasks | Graph Embedding, Knowledge Base Completion, Knowledge Graph Completion, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction, Relational Reasoning |
| Published | 2019-12-31 |
| URL | https://ieeexplore.ieee.org/abstract/document/8946600 |
| https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8946600 | |
| PWC | https://paperswithcode.com/paper/knowledge-graph-embedding-via-graph |
| Repo | |
| Framework | |
Inverse Procedural Modeling of Knitwear
| Title | Inverse Procedural Modeling of Knitwear |
| Authors | Elena Trunz, Sebastian Merzbach, Jonathan Klein, Thomas Schulze, Michael Weinmann, Reinhard Klein |
| Abstract | The analysis and modeling of cloth has received a lot of attention in recent years. While recent approaches are focused on woven cloth, we present a novel practical approach for the inference of more complex knitwear structures as well as the respective knitting instructions from only a single image without attached annotations. Knitwear is produced by repeating instances of the same pattern, consisting of grid-like arrangements of a small set of basic stitch types. Our framework addresses the identification and localization of the occurring stitch types, which is challenging due to huge appearance variations. The resulting coarsely localized stitch types are used to infer the underlying grid structure as well as for the extraction of the knitting instruction of pattern repeats, taking into account principles of Gestalt theory. Finally, the derived instructions allow the reproduction of the knitting structures, either as renderings or by actual knitting, as demonstrated in several examples. |
| Tasks | |
| Published | 2019-06-01 |
| URL | http://openaccess.thecvf.com/content_CVPR_2019/html/Trunz_Inverse_Procedural_Modeling_of_Knitwear_CVPR_2019_paper.html |
| http://openaccess.thecvf.com/content_CVPR_2019/papers/Trunz_Inverse_Procedural_Modeling_of_Knitwear_CVPR_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/inverse-procedural-modeling-of-knitwear |
| Repo | |
| Framework | |
DISTRIBUTIONAL CONCAVITY REGULARIZATION FOR GANS
| Title | DISTRIBUTIONAL CONCAVITY REGULARIZATION FOR GANS |
| Authors | Shoichiro Yamaguchi, Masanori Koyama |
| Abstract | We propose Distributional Concavity (DC) regularization for Generative Adversarial Networks (GANs), a functional gradient-based method that promotes the entropy of the generator distribution and works against mode collapse. Our DC regularization is an easy-to-implement method that can be used in combination with the current state of the art methods like Spectral Normalization and Wasserstein GAN with gradient penalty to further improve the performance. We will not only show that our DC regularization can achieve highly competitive results on ILSVRC2012 and CIFAR datasets in terms of Inception score and Fr'echet inception distance, but also provide a mathematical guarantee that our method can always increase the entropy of the generator distribution. We will also show an intimate theoretical connection between our method and the theory of optimal transport. |
| Tasks | |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=SklEEnC5tQ |
| https://openreview.net/pdf?id=SklEEnC5tQ | |
| PWC | https://paperswithcode.com/paper/distributional-concavity-regularization-for |
| Repo | |
| Framework | |
CDeepEx: Contrastive Deep Explanations
| Title | CDeepEx: Contrastive Deep Explanations |
| Authors | Amir Feghahati, Christian R. Shelton, Michael J. Pazzani, Kevin Tang |
| Abstract | We propose a method which can visually explain the classification decision of deep neural networks (DNNs). There are many proposed methods in machine learning and computer vision seeking to clarify the decision of machine learning black boxes, specifically DNNs. All of these methods try to gain insight into why the network “chose class A” as an answer. Humans, when searching for explanations, ask two types of questions. The first question is, “Why did you choose this answer?” The second question asks, “Why did you not choose answer B over A?” The previously proposed methods are either not able to provide the latter directly or efficiently. We introduce a method capable of answering the second question both directly and efficiently. In this work, we limit the inputs to be images. In general, the proposed method generates explanations in the input space of any model capable of efficient evaluation and gradient evaluation. We provide results, showing the superiority of this approach for gaining insight into the inner representation of machine learning models. |
| Tasks | |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=HyNmRiCqtm |
| https://openreview.net/pdf?id=HyNmRiCqtm | |
| PWC | https://paperswithcode.com/paper/cdeepex-contrastive-deep-explanations |
| Repo | |
| Framework | |
Dynamically Composing Domain-Data Selection with Clean-Data Selection by ``Co-Curricular Learning’’ for Neural Machine Translation
| Title | Dynamically Composing Domain-Data Selection with Clean-Data Selection by ``Co-Curricular Learning’’ for Neural Machine Translation | |
| Authors | Wei Wang, Isaac Caswell, Ciprian Chelba |
| Abstract | Noise and domain are important aspects of data quality for neural machine translation. Existing research focus separately on domain-data selection, clean-data selection, or their static combination, leaving the dynamic interaction across them not explicitly examined. This paper introduces a {}co-curricular learning{''} method to compose dynamic domain-data selection with dynamic clean-data selection, for transfer learning across both capabilities. We apply an EM-style optimization procedure to further refine the {}co-curriculum{''}. Experiment results and analysis with two domains demonstrate the effectiveness of the method and the properties of data scheduled by the co-curriculum. |
| Tasks | Machine Translation, Transfer Learning |
| Published | 2019-07-01 |
| URL | https://www.aclweb.org/anthology/P19-1123/ |
| https://www.aclweb.org/anthology/P19-1123 | |
| PWC | https://paperswithcode.com/paper/dynamically-composing-domain-data-selection-1 |
| Repo | |
| Framework | |
Head-First Linearization with Tree-Structured Representation
| Title | Head-First Linearization with Tree-Structured Representation |
| Authors | Xiang Yu, Agnieszka Falenska, Ngoc Thang Vu, Jonas Kuhn |
| Abstract | We present a dependency tree linearization model with two novel components: (1) a tree-structured encoder based on bidirectional Tree-LSTM that propagates information first bottom-up then top-down, which allows each token to access information from the entire tree; and (2) a linguistically motivated head-first decoder that emphasizes the central role of the head and linearizes the subtree by incrementally attaching the dependents on both sides of the head. With the new encoder and decoder, we reach state-of-the-art performance on the Surface Realization Shared Task 2018 dataset, outperforming not only the shared tasks participants, but also previous state-of-the-art systems (Bohnet et al., 2011; Puduppully et al., 2016). Furthermore, we analyze the power of the tree-structured encoder with a probing task and show that it is able to recognize the topological relation between any pair of tokens in a tree. |
| Tasks | |
| Published | 2019-10-01 |
| URL | https://www.aclweb.org/anthology/W19-8636/ |
| https://www.aclweb.org/anthology/W19-8636 | |
| PWC | https://paperswithcode.com/paper/head-first-linearization-with-tree-structured |
| Repo | |
| Framework | |
Adversarial Fine-Grained Composition Learning for Unseen Attribute-Object Recognition
| Title | Adversarial Fine-Grained Composition Learning for Unseen Attribute-Object Recognition |
| Authors | Kun Wei, Muli Yang, Hao Wang, Cheng Deng, Xianglong Liu |
| Abstract | Recognizing unseen attribute-object pairs never appearing in the training data is a challenging task, since an object often refers to a specific entity while an attribute is an abstract semantic description. Besides, attributes are highly correlated to objects, i.e., an attribute tends to describe different visual features of various objects. Existing methods mainly employ two classifiers to recognize attribute and object separately, or simply simulate the composition of attribute and object, which ignore the inherent discrepancy and correlation between them. In this paper, we propose a novel adversarial fine-grained composition learning model for unseen attribute-object pair recognition. Considering their inherent discrepancy, we leverage multi-scale feature integration to capture discriminative fine-grained features from a given image. Besides, we devise a quintuplet loss to depict more accurate correlations between attributes and objects. Adversarial learning is employed to model the discrepancy and correlations among attributes and objects. Extensive experiments on two challenging benchmarks indicate that our method consistently outperforms state-of-the-art competitors by a large margin. |
| Tasks | Object Recognition |
| Published | 2019-10-01 |
| URL | http://openaccess.thecvf.com/content_ICCV_2019/html/Wei_Adversarial_Fine-Grained_Composition_Learning_for_Unseen_Attribute-Object_Recognition_ICCV_2019_paper.html |
| http://openaccess.thecvf.com/content_ICCV_2019/papers/Wei_Adversarial_Fine-Grained_Composition_Learning_for_Unseen_Attribute-Object_Recognition_ICCV_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-fine-grained-composition-learning |
| Repo | |
| Framework | |
Unsupervised Cross-Lingual Representation Learning
| Title | Unsupervised Cross-Lingual Representation Learning |
| Authors | Sebastian Ruder, Anders S{\o}gaard, Ivan Vuli{'c} |
| Abstract | In this tutorial, we provide a comprehensive survey of the exciting recent work on cutting-edge weakly-supervised and unsupervised cross-lingual word representations. After providing a brief history of supervised cross-lingual word representations, we focus on: 1) how to induce weakly-supervised and unsupervised cross-lingual word representations in truly resource-poor settings where bilingual supervision cannot be guaranteed; 2) critical examinations of different training conditions and requirements under which unsupervised algorithms can and cannot work effectively; 3) more robust methods for distant language pairs that can mitigate instability issues and low performance for distant language pairs; 4) how to comprehensively evaluate such representations; and 5) diverse applications that benefit from cross-lingual word representations (e.g., MT, dialogue, cross-lingual sequence labeling and structured prediction applications, cross-lingual IR). |
| Tasks | Representation Learning, Structured Prediction |
| Published | 2019-07-01 |
| URL | https://www.aclweb.org/anthology/P19-4007/ |
| https://www.aclweb.org/anthology/P19-4007 | |
| PWC | https://paperswithcode.com/paper/unsupervised-cross-lingual-representation |
| Repo | |
| Framework | |
Proceedings of the 3rd Workshop on the Use of Computational Methods in the Study of Endangered Languages Volume 1 (Papers)
| Title | Proceedings of the 3rd Workshop on the Use of Computational Methods in the Study of Endangered Languages Volume 1 (Papers) |
| Authors | |
| Abstract | |
| Tasks | |
| Published | 2019-02-01 |
| URL | https://www.aclweb.org/anthology/W19-6000/ |
| https://www.aclweb.org/anthology/W19-6000 | |
| PWC | https://paperswithcode.com/paper/proceedings-of-the-3rd-workshop-on-the-use-of |
| Repo | |
| Framework | |
SEGEN: SAMPLE-ENSEMBLE GENETIC EVOLUTIONARY NETWORK MODEL
| Title | SEGEN: SAMPLE-ENSEMBLE GENETIC EVOLUTIONARY NETWORK MODEL |
| Authors | Jiawei Zhang, Limeng Cui, Fisher B. Gouza |
| Abstract | Deep learning, a rebranding of deep neural network research works, has achieved a remarkable success in recent years. With multiple hidden layers, deep learning models aim at computing the hierarchical feature representations of the observational data. Meanwhile, due to its severe disadvantages in data consumption, computational resources, parameter tuning costs and the lack of result explainability, deep learning has also suffered from lots of criticism. In this paper, we will introduce a new representation learning model, namely “Sample-Ensemble Genetic Evolutionary Network” (SEGEN), which can serve as an alternative approach to deep learning models. Instead of building one single deep model, based on a set of sampled sub-instances, SEGEN adopts a genetic-evolutionary learning strategy to build a group of unit models generations by generations. The unit models incorporated in SEGEN can be either traditional machine learning models or the recent deep learning models with a much “narrower” and “shallower” architecture. The learning results of each instance at the final generation will be effectively combined from each unit model via diffusive propagation and ensemble learning strategies. From the computational perspective, SEGEN requires far less data, fewer computational resources and parameter tuning efforts, but has sound theoretic interpretability of the learning process and results. Extensive experiments have been done on several different real-world benchmark datasets, and the experimental results obtained by SEGEN have demonstrated its advantages over the state-of-the-art representation learning models. |
| Tasks | Representation Learning |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=HJgVisRqtX |
| https://openreview.net/pdf?id=HJgVisRqtX | |
| PWC | https://paperswithcode.com/paper/segen-sample-ensemble-genetic-evolutionary |
| Repo | |
| Framework | |
Explaining Simple Natural Language Inference
| Title | Explaining Simple Natural Language Inference |
| Authors | Aikaterini-Lida Kalouli, Annebeth Buis, Livy Real, Martha Palmer, Valeria de Paiva |
| Abstract | The vast amount of research introducing new corpora and techniques for semi-automatically annotating corpora shows the important role that datasets play in today{'}s research, especially in the machine learning community. This rapid development raises concerns about the quality of the datasets created and consequently of the models trained, as recently discussed with respect to the Natural Language Inference (NLI) task. In this work we conduct an annotation experiment based on a small subset of the SICK corpus. The experiment reveals several problems in the annotation guidelines, and various challenges of the NLI task itself. Our quantitative evaluation of the experiment allows us to assign our empirical observations to specific linguistic phenomena and leads us to recommendations for future annotation tasks, for NLI and possibly for other tasks. |
| Tasks | Natural Language Inference |
| Published | 2019-08-01 |
| URL | https://www.aclweb.org/anthology/W19-4016/ |
| https://www.aclweb.org/anthology/W19-4016 | |
| PWC | https://paperswithcode.com/paper/explaining-simple-natural-language-inference |
| Repo | |
| Framework | |
A theoretical framework for deep and locally connected ReLU network
| Title | A theoretical framework for deep and locally connected ReLU network |
| Authors | Yuandong Tian |
| Abstract | Understanding theoretical properties of deep and locally connected nonlinear network, such as deep convolutional neural network (DCNN), is still a hard problem despite its empirical success. In this paper, we propose a novel theoretical framework for such networks with ReLU nonlinearity. The framework bridges data distribution with gradient descent rules, favors disentangled representations and is compatible with common regularization techniques such as Batch Norm, after a novel discovery of its projection nature. The framework is built upon teacher-student setting, by projecting the student’s forward/backward pass onto the teacher’s computational graph. We do not impose unrealistic assumptions (e.g., Gaussian inputs, independence of activation, etc). Our framework could help facilitate theoretical analysis of many practical issues, e.g. disentangled representations in deep networks. |
| Tasks | |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=SyeKf30cFQ |
| https://openreview.net/pdf?id=SyeKf30cFQ | |
| PWC | https://paperswithcode.com/paper/a-theoretical-framework-for-deep-and-locally |
| Repo | |
| Framework | |
Analyzing Linguistic Complexity and Accuracy in Academic Language Development of German across Elementary and Secondary School
| Title | Analyzing Linguistic Complexity and Accuracy in Academic Language Development of German across Elementary and Secondary School |
| Authors | Zarah Weiss, Detmar Meurers |
| Abstract | We track the development of writing complexity and accuracy in German students{'} early academic language development from first to eighth grade. Combining an empirically broad approach to linguistic complexity with the high-quality error annotation included in the Karlsruhe Children{'}s Text corpus (Lavalley et al. 2015) used, we construct models of German academic language development that successfully identify the student{'}s grade level. We show that classifiers for the early years rely more on accuracy development, whereas development in secondary school is better characterized by increasingly complex language in all domains: linguistic system, language use, and human sentence processing characteristics. We demonstrate the generalizability and robustness of models using such a broad complexity feature set across writing topics. |
| Tasks | |
| Published | 2019-08-01 |
| URL | https://www.aclweb.org/anthology/W19-4440/ |
| https://www.aclweb.org/anthology/W19-4440 | |
| PWC | https://paperswithcode.com/paper/analyzing-linguistic-complexity-and-accuracy |
| Repo | |
| Framework | |
Transferring SLU Models in Novel Domains
| Title | Transferring SLU Models in Novel Domains |
| Authors | Yaohua Tang, Kaixiang Mo, Qian Xu, Chao Zhang, Qiang Yang |
| Abstract | Spoken language understanding (SLU) is a critical component in building dialogue systems. When building models for novel natural language domains, a major challenge is the lack of data in the new domains, no matter whether the data is annotated or not. Recognizing and annotating intent'' and slot’’ of natural languages is a time-consuming process. Therefore, spoken language understanding in low resource domains remains a crucial problem to address. In this paper, we address this problem by proposing a transfer-learning method, whereby a SLU model is transferred to a novel but data-poor domain via a deep neural network framework. We also introduce meta-learning in our work to bridge the semantic relations between seen and unseen data, allowing new intents to be recognized and new slots to be filled with much lower new training effort. We show the performance improvement with extensive experimental results for spoken language understanding in low resource domains. We show that our method can also handle novel intent recognition and slot-filling tasks. Our methodology provides a feasible solution for alleviating data shortages in spoken language understanding. |
| Tasks | Meta-Learning, Slot Filling, Spoken Language Understanding, Transfer Learning |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=rkg5fh0ctQ |
| https://openreview.net/pdf?id=rkg5fh0ctQ | |
| PWC | https://paperswithcode.com/paper/transferring-slu-models-in-novel-domains |
| Repo | |
| Framework | |
