Paper Group NANR 260
A Meta-Analysis of Overfitting in Machine Learning. Multi-lingual and Cross-genre Discourse Unit Segmentation. DOMLIN at SemEval-2019 Task 8: Automated Fact Checking exploiting Ratings in Community Question Answering Forums. YNU-HPCC at SemEval-2019 Task 8: Using A LSTM-Attention Model for Fact-Checking in Community Forums. ClusterNet: Deep Hierarc …
A Meta-Analysis of Overfitting in Machine Learning
| Title | A Meta-Analysis of Overfitting in Machine Learning |
| Authors | Rebecca Roelofs, Vaishaal Shankar, Benjamin Recht, Sara Fridovich-Keil, Moritz Hardt, John Miller, Ludwig Schmidt |
| Abstract | We conduct the first large meta-analysis of overfitting due to test set reuse in the machine learning community. Our analysis is based on over one hundred machine learning competitions hosted on the Kaggle platform over the course of several years. In each competition, numerous practitioners repeatedly evaluated their progress against a holdout set that forms the basis of a public ranking available throughout the competition. Performance on a separate test set used only once determined the final ranking. By systematically comparing the public ranking with the final ranking, we assess how much participants adapted to the holdout set over the course of a competition. Our study shows, somewhat surprisingly, little evidence of substantial overfitting. These findings speak to the robustness of the holdout method across different data domains, loss functions, model classes, and human analysts. |
| Tasks | |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9117-a-meta-analysis-of-overfitting-in-machine-learning |
| http://papers.nips.cc/paper/9117-a-meta-analysis-of-overfitting-in-machine-learning.pdf | |
| PWC | https://paperswithcode.com/paper/a-meta-analysis-of-overfitting-in-machine |
| Repo | |
| Framework | |
Multi-lingual and Cross-genre Discourse Unit Segmentation
| Title | Multi-lingual and Cross-genre Discourse Unit Segmentation |
| Authors | Peter Bourgonje, Robin Sch{"a}fer |
| Abstract | We describe a series of experiments applied to data sets from different languages and genres annotated for coherence relations according to different theoretical frameworks. Specifically, we investigate the feasibility of a unified (theory-neutral) approach toward discourse segmentation; a process which divides a text into minimal discourse units that are involved in s coherence relation. We apply a RandomForest and an LSTM based approach for all data sets, and we improve over a simple baseline assuming simple sentence or clause-like segmentation. Performance however varies a lot depending on language, and more importantly genre, with f-scores ranging from 73.00 to 94.47. |
| Tasks | |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/W19-2714/ |
| https://www.aclweb.org/anthology/W19-2714 | |
| PWC | https://paperswithcode.com/paper/multi-lingual-and-cross-genre-discourse-unit |
| Repo | |
| Framework | |
DOMLIN at SemEval-2019 Task 8: Automated Fact Checking exploiting Ratings in Community Question Answering Forums
| Title | DOMLIN at SemEval-2019 Task 8: Automated Fact Checking exploiting Ratings in Community Question Answering Forums |
| Authors | Dominik Stammbach, Stalin Varanasi, Guenter Neumann |
| Abstract | In the following, we describe our system developed for the Semeval2019 Task 8. We fine-tuned a BERT checkpoint on the qatar living forum dump and used this checkpoint to train a number of models. Our hand-in for subtask A consists of a fine-tuned classifier from this BERT checkpoint. For subtask B, we first have a classifier deciding whether a comment is factual or non-factual. If it is factual, we retrieve intra-forum evidence and using this evidence, have a classifier deciding the comment{'}s veracity. We trained this classifier on ratings which we crawled from qatarliving.com |
| Tasks | Community Question Answering, Question Answering |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2201/ |
| https://www.aclweb.org/anthology/S19-2201 | |
| PWC | https://paperswithcode.com/paper/domlin-at-semeval-2019-task-8-automated-fact |
| Repo | |
| Framework | |
YNU-HPCC at SemEval-2019 Task 8: Using A LSTM-Attention Model for Fact-Checking in Community Forums
| Title | YNU-HPCC at SemEval-2019 Task 8: Using A LSTM-Attention Model for Fact-Checking in Community Forums |
| Authors | Peng Liu, Jin Wang, Xuejie Zhang |
| Abstract | We propose a system that uses a long short-term memory with attention mechanism (LSTM-Attention) model to complete the task. The LSTM-Attention model uses two LSTM to extract the features of the question and answer pair. Then, each of the features is sequentially composed using the attention mechanism, concatenating the two vectors into one. Finally, the concatenated vector is used as input for the MLP and the MLP{'}s output layer uses the softmax function to classify the provided answers into three categories. This model is capable of extracting the features of the question and answer pair well. The results show that the proposed system outperforms the baseline algorithm. |
| Tasks | |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2207/ |
| https://www.aclweb.org/anthology/S19-2207 | |
| PWC | https://paperswithcode.com/paper/ynu-hpcc-at-semeval-2019-task-8-using-a-lstm |
| Repo | |
| Framework | |
ClusterNet: Deep Hierarchical Cluster Network With Rigorously Rotation-Invariant Representation for Point Cloud Analysis
| Title | ClusterNet: Deep Hierarchical Cluster Network With Rigorously Rotation-Invariant Representation for Point Cloud Analysis |
| Authors | Chao Chen, Guanbin Li, Ruijia Xu, Tianshui Chen, Meng Wang, Liang Lin |
| Abstract | Current neural networks for 3D object recognition are vulnerable to 3D rotation. Existing works mostly rely on massive amounts of rotation-augmented data to alleviate the problem, which lacks solid guarantee of the 3D rotation invariance. In this paper, we address the issue by introducing a novel point cloud representation that can be mathematically proved rigorously rotation-invariant, i.e., identical point clouds in different orientations are unified as a unique and consistent representation. Moreover, the proposed representation is conditional information-lossless, because it retains all necessary information of point cloud except for orientation information. In addition, the proposed representation is complementary with existing network architectures for point cloud and fundamentally improves their robustness against rotation transformation. Finally, we propose a deep hierarchical cluster network called ClusterNet to better adapt to the proposed representation. We employ hierarchical clustering to explore and exploit the geometric structure of point cloud, which is embedded in a hierarchical structure tree. Extensive experimental results have shown that our proposed method greatly outperforms the state-of-the-arts in rotation robustness on rotation-augmented 3D object classification benchmarks. |
| Tasks | 3D Object Classification, 3D Object Recognition, Object Classification, Object Recognition |
| Published | 2019-06-01 |
| URL | http://openaccess.thecvf.com/content_CVPR_2019/html/Chen_ClusterNet_Deep_Hierarchical_Cluster_Network_With_Rigorously_Rotation-Invariant_Representation_for_CVPR_2019_paper.html |
| http://openaccess.thecvf.com/content_CVPR_2019/papers/Chen_ClusterNet_Deep_Hierarchical_Cluster_Network_With_Rigorously_Rotation-Invariant_Representation_for_CVPR_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/clusternet-deep-hierarchical-cluster-network |
| Repo | |
| Framework | |
Diachronic Analysis of Entities by Exploiting Wikipedia Page revisions
| Title | Diachronic Analysis of Entities by Exploiting Wikipedia Page revisions |
| Authors | Pierpaolo Basile, Annalina Caputo, Seamus Lawless, Giovanni Semeraro |
| Abstract | In the last few years, the increasing availability of large corpora spanning several time periods has opened new opportunities for the diachronic analysis of language. This type of analysis can bring to the light not only linguistic phenomena related to the shift of word meanings over time, but it can also be used to study the impact that societal and cultural trends have on this language change. This paper introduces a new resource for performing the diachronic analysis of named entities built upon Wikipedia page revisions. This resource enables the analysis over time of changes in the relations between entities (concepts), surface forms (words), and the contexts surrounding entities and surface forms, by analysing the whole history of Wikipedia internal links. We provide some useful use cases that prove the impact of this resource on diachronic studies and delineate some possible future usage. |
| Tasks | |
| Published | 2019-09-01 |
| URL | https://www.aclweb.org/anthology/R19-1011/ |
| https://www.aclweb.org/anthology/R19-1011 | |
| PWC | https://paperswithcode.com/paper/diachronic-analysis-of-entities-by-exploiting |
| Repo | |
| Framework | |
Metamers of neural networks reveal divergence from human perceptual systems
| Title | Metamers of neural networks reveal divergence from human perceptual systems |
| Authors | Jenelle Feather, Alex Durango, Ray Gonzalez, Josh Mcdermott |
| Abstract | Deep neural networks have been embraced as models of sensory systems, instantiating representational transformations that appear to resemble those in the visual and auditory systems. To more thoroughly investigate their similarity to biological systems, we synthesized model metamers – stimuli that produce the same responses at some stage of a network’s representation. We generated model metamers for natural stimuli by performing gradient descent on a noise signal, matching the responses of individual layers of image and audio networks to a natural image or speech signal. The resulting signals reflect the invariances instantiated in the network up to the matched layer. We then measured whether model metamers were recognizable to human observers – a necessary condition for the model representations to replicate those of humans. Although model metamers from early network layers were recognizable to humans, those from deeper layers were not. Auditory model metamers became more human-recognizable with architectural modifications that reduced aliasing from pooling operations, but those from the deepest layers remained unrecognizable. We also used the metamer test to compare model representations. Cross-model metamer recognition dropped off for deeper layers, roughly at the same point that human recognition deteriorated, indicating divergence across model representations. The results reveal discrepancies between model and human representations, but also show how metamers can help guide model refinement and elucidate model representations. |
| Tasks | |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9198-metamers-of-neural-networks-reveal-divergence-from-human-perceptual-systems |
| http://papers.nips.cc/paper/9198-metamers-of-neural-networks-reveal-divergence-from-human-perceptual-systems.pdf | |
| PWC | https://paperswithcode.com/paper/metamers-of-neural-networks-reveal-divergence |
| Repo | |
| Framework | |
De-Mixing Sentiment from Code-Mixed Text
| Title | De-Mixing Sentiment from Code-Mixed Text |
| Authors | Yash Kumar Lal, Vaibhav Kumar, Mrinal Dhar, Manish Shrivastava, Philipp Koehn |
| Abstract | Code-mixing is the phenomenon of mixing the vocabulary and syntax of multiple languages in the same sentence. It is an increasingly common occurrence in today{'}s multilingual society and poses a big challenge when encountered in different downstream tasks. In this paper, we present a hybrid architecture for the task of Sentiment Analysis of English-Hindi code-mixed data. Our method consists of three components, each seeking to alleviate different issues. We first generate subword level representations for the sentences using a CNN architecture. The generated representations are used as inputs to a Dual Encoder Network which consists of two different BiLSTMs - the Collective and Specific Encoder. The Collective Encoder captures the overall sentiment of the sentence, while the Specific Encoder utilizes an attention mechanism in order to focus on individual sentiment-bearing sub-words. This, combined with a Feature Network consisting of orthographic features and specially trained word embeddings, achieves state-of-the-art results - 83.54{%} accuracy and 0.827 F1 score - on a benchmark dataset. |
| Tasks | Sentiment Analysis, Word Embeddings |
| Published | 2019-07-01 |
| URL | https://www.aclweb.org/anthology/P19-2052/ |
| https://www.aclweb.org/anthology/P19-2052 | |
| PWC | https://paperswithcode.com/paper/de-mixing-sentiment-from-code-mixed-text |
| Repo | |
| Framework | |
OleNet at SemEval-2019 Task 9: BERT based Multi-Perspective Models for Suggestion Mining
| Title | OleNet at SemEval-2019 Task 9: BERT based Multi-Perspective Models for Suggestion Mining |
| Authors | Jiaxiang Liu, Shuohuan Wang, Yu Sun |
| Abstract | This paper describes our system partici- pated in Task 9 of SemEval-2019: the task is focused on suggestion mining and it aims to classify given sentences into sug- gestion and non-suggestion classes in do- main specific and cross domain training setting respectively. We propose a multi- perspective architecture for learning rep- resentations by using different classical models including Convolutional Neural Networks (CNN), Gated Recurrent Units (GRU), Feed Forward Attention (FFA), etc. To leverage the semantics distributed in large amount of unsupervised data, we also have adopted the pre-trained Bidi- rectional Encoder Representations from Transformers (BERT) model as an en- coder to produce sentence and word rep- resentations. The proposed architecture is applied for both sub-tasks, and achieved f1-score of 0.7812 for subtask A, and 0.8579 for subtask B. We won the first and second place for the two tasks respec- tively in the final competition. |
| Tasks | |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2216/ |
| https://www.aclweb.org/anthology/S19-2216 | |
| PWC | https://paperswithcode.com/paper/olenet-at-semeval-2019-task-9-bert-based |
| Repo | |
| Framework | |
SSN-SPARKS at SemEval-2019 Task 9: Mining Suggestions from Online Reviews using Deep Learning Techniques on Augmented Data
| Title | SSN-SPARKS at SemEval-2019 Task 9: Mining Suggestions from Online Reviews using Deep Learning Techniques on Augmented Data |
| Authors | Rajalakshmi S, Angel Suseelan, S Milton Rajendram, Mirnalinee T T |
| Abstract | This paper describes the work on mining the suggestions from online reviews and forums. Opinion mining detects whether the comments are positive, negative or neutral, while suggestion mining explores the review content for the possible tips or advice. The system developed by SSN-SPARKS team in SemEval-2019 for task 9 (suggestion mining) uses a rule-based approach for feature selection, SMOTE technique for data augmentation and deep learning technique (Convolutional Neural Network) for classification. We have compared the results with Random Forest classifier (RF) and MultiLayer Perceptron (MLP) model. Results show that the CNN model performs better than other models for both the subtasks. |
| Tasks | Data Augmentation, Feature Selection, Opinion Mining |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2217/ |
| https://www.aclweb.org/anthology/S19-2217 | |
| PWC | https://paperswithcode.com/paper/ssn-sparks-at-semeval-2019-task-9-mining |
| Repo | |
| Framework | |
A Dynamic Semantics for Causal Counterfactuals
| Title | A Dynamic Semantics for Causal Counterfactuals |
| Authors | Kenneth Lai, James Pustejovsky |
| Abstract | Under the standard approach to counterfactuals, to determine the meaning of a counterfactual sentence, we consider the {}closest{''} possible world(s) where the antecedent is true, and evaluate the consequent. Building on the standard approach, some researchers have found that the set of worlds to be considered is dependent on context; it evolves with the discourse. Others have focused on how to define the {}distance{''} between possible worlds, using ideas from causal modeling. This paper integrates the two ideas. We present a semantics for counterfactuals that uses a distance measure based on causal laws, that can also change over time. We show how our semantics can be implemented in the Haskell programming language. |
| Tasks | |
| Published | 2019-05-01 |
| URL | https://www.aclweb.org/anthology/W19-0601/ |
| https://www.aclweb.org/anthology/W19-0601 | |
| PWC | https://paperswithcode.com/paper/a-dynamic-semantics-for-causal |
| Repo | |
| Framework | |
Suggestion Miner at SemEval-2019 Task 9: Suggestion Detection in Online Forum using Word Graph
| Title | Suggestion Miner at SemEval-2019 Task 9: Suggestion Detection in Online Forum using Word Graph |
| Authors | Usman Ahmed, Humera Liaquat, Luqman Ahmed, Syed Jawad Hussain |
| Abstract | This paper describes the suggestion miner system that participates in SemEval 2019 Task 9 - SubTask A - Suggestion Mining from Online Reviews and Forums. The system participated in the subtasks A. This paper discusses the results of our system in the development, evaluation and post evaluation. Each class in the dataset is represented as directed unweighted graphs. Then, the comparison is carried out with each class graph which results in a vector. This vector is used as features by a machine learning algorithm. The model is evaluated on hold on strategy. The organizers randomly split (8500 instances) training set (provided to the participant in training their system) and testing set (833 instances). The test set is reserved to evaluate the performance of participants systems. During the evaluation, our system ranked 31 in the Coda Lab result of the subtask A (binary class problem). The binary class system achieves evaluation value 0.34, precision 0.87, recall 0.73 and F measure 0.78. |
| Tasks | |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2218/ |
| https://www.aclweb.org/anthology/S19-2218 | |
| PWC | https://paperswithcode.com/paper/suggestion-miner-at-semeval-2019-task-9 |
| Repo | |
| Framework | |
YNU-HPCC at SemEval-2019 Task 9: Using a BERT and CNN-BiLSTM-GRU Model for Suggestion Mining
| Title | YNU-HPCC at SemEval-2019 Task 9: Using a BERT and CNN-BiLSTM-GRU Model for Suggestion Mining |
| Authors | Ping Yue, Jin Wang, Xuejie Zhang |
| Abstract | Consumer opinions towards commercial entities are generally expressed through online reviews, blogs, and discussion forums. These opinions largely express positive and negative sentiments towards a given entity,but also tend to contain suggestions for improving the entity. In this task, we extract suggestions from given the unstructured text, compared to the traditional opinion mining systems. Such suggestion mining is more applicability and extends capabilities. |
| Tasks | Opinion Mining |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2224/ |
| https://www.aclweb.org/anthology/S19-2224 | |
| PWC | https://paperswithcode.com/paper/ynu-hpcc-at-semeval-2019-task-9-using-a-bert |
| Repo | |
| Framework | |
ProblemSolver at SemEval-2019 Task 10: Sequence-to-Sequence Learning and Expression Trees
| Title | ProblemSolver at SemEval-2019 Task 10: Sequence-to-Sequence Learning and Expression Trees |
| Authors | Xuefeng Luo, Alina Baranova, Jonas Biegert |
| Abstract | This paper describes our participation in SemEval-2019 shared task {}Math Question Answering{''}, where the aim is to create a program that could solve the Math SAT questions automatically as accurately as possible. We went with a dual-pronged approach, building a Sequence-to-Sequence Neural Network pre-trained with augmented data that could answer all categories of questions and a Tree system, which can only answer a certain type of questions. The systems did not perform well on the entire test data given in the task, but did decently on the questions they were actually capable of answering. The Sequence-to-Sequence Neural Network model managed to get slightly better than our baseline of guessing {}A{''} for every question, while the Tree system additionally improved the results. |
| Tasks | Question Answering |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2227/ |
| https://www.aclweb.org/anthology/S19-2227 | |
| PWC | https://paperswithcode.com/paper/problemsolver-at-semeval-2019-task-10 |
| Repo | |
| Framework | |
RGCL-WLV at SemEval-2019 Task 12: Toponym Detection
| Title | RGCL-WLV at SemEval-2019 Task 12: Toponym Detection |
| Authors | Alistair Plum, Tharindu Ranasinghe, Pablo Calleja, Constantin Or{\u{a}}san, Ruslan Mitkov |
| Abstract | This article describes the system submitted by the RGCL-WLV team to the SemEval 2019 Task 12: Toponym resolution in scientific papers. The system detects toponyms using a bootstrapped machine learning (ML) approach which classifies names identified using gazetteers extracted from the GeoNames geographical database. The paper evaluates the performance of several ML classifiers, as well as how the gazetteers influence the accuracy of the system. Several runs were submitted. The highest precision achieved for one of the submissions was 89{%}, albeit it at a relatively low recall of 49{%}. |
| Tasks | |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2228/ |
| https://www.aclweb.org/anthology/S19-2228 | |
| PWC | https://paperswithcode.com/paper/rgcl-wlv-at-semeval-2019-task-12-toponym |
| Repo | |
| Framework | |
