Paper Group NANR 151
In Your Pace: Learning the Right Example at the Right Time. Twitter Bot Detection using Diversity Measures. Analysing Representations of Memory Impairment in a Clinical Notes Classification Model. Rumor Detection by Exploiting User Credibility Information, Attention and Multi-task Learning. Occlusion-Aware Networks for 3D Human Pose Estimation in V …
In Your Pace: Learning the Right Example at the Right Time
| Title | In Your Pace: Learning the Right Example at the Right Time |
| Authors | Guy Hacohen, Daphna Weinshall |
| Abstract | Training neural networks is traditionally done by sequentially providing random mini-batches sampled uniformly from the entire dataset. In our work, we show that sampling mini-batches non-uniformly can both enhance the speed of learning and improve the final accuracy of the trained network. Specifically, we decompose the problem using the principles of curriculum learning: first, we sort the data by some difficulty measure; second, we sample mini-batches with a gradually increasing level of difficulty. We focus on CNNs trained on image recognition. Initially, we define the difficulty of a training image using transfer learning from some competitive “teacher” network trained on the Imagenet database, showing improvement in learning speed and final performance for both small and competitive networks, using the CIFAR-10 and the CIFAR-100 datasets. We then suggest a bootstrap alternative to evaluate the difficulty of points using the same network without relying on a “teacher” network, thus increasing the applicability of our suggested method. We compare this approach to a related version of Self-Paced Learning, showing that our method benefits learning while SPL impairs it. |
| Tasks | Transfer Learning |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=ryxHii09KQ |
| https://openreview.net/pdf?id=ryxHii09KQ | |
| PWC | https://paperswithcode.com/paper/in-your-pace-learning-the-right-example-at |
| Repo | |
| Framework | |
Twitter Bot Detection using Diversity Measures
| Title | Twitter Bot Detection using Diversity Measures |
| Authors | Dijana Kosmajac, Vlado Keselj |
| Abstract | |
| Tasks | Twitter Bot Detection |
| Published | 2019-09-01 |
| URL | https://www.aclweb.org/anthology/W19-7401/ |
| https://www.aclweb.org/anthology/W19-7401 | |
| PWC | https://paperswithcode.com/paper/twitter-bot-detection-using-diversity |
| Repo | |
| Framework | |
Analysing Representations of Memory Impairment in a Clinical Notes Classification Model
| Title | Analysing Representations of Memory Impairment in a Clinical Notes Classification Model |
| Authors | Mark Ormerod, Jes{'u}s Mart{'\i}nez-del-Rinc{'o}n, Neil Robertson, Bernadette McGuinness, Barry Devereux |
| Abstract | Despite recent advances in the application of deep neural networks to various kinds of medical data, extracting information from unstructured textual sources remains a challenging task. The challenges of training and interpreting document classification models are amplified when dealing with small and highly technical datasets, as are common in the clinical domain. Using a dataset of de-identified clinical letters gathered at a memory clinic, we construct several recurrent neural network models for letter classification, and evaluate them on their ability to build meaningful representations of the documents and predict patients{'} diagnoses. Additionally, we probe sentence embedding models in order to build a human-interpretable representation of the neural network{'}s features, using a simple and intuitive technique based on perturbative approaches to sentence importance. In addition to showing which sentences in a document are most informative about the patient{'}s condition, this method reveals the types of sentences that lead the model to make incorrect diagnoses. Furthermore, we identify clusters of sentences in the embedding space that correlate strongly with importance scores for each clinical diagnosis class. |
| Tasks | Document Classification, Sentence Embedding |
| Published | 2019-08-01 |
| URL | https://www.aclweb.org/anthology/W19-5005/ |
| https://www.aclweb.org/anthology/W19-5005 | |
| PWC | https://paperswithcode.com/paper/analysing-representations-of-memory |
| Repo | |
| Framework | |
Rumor Detection by Exploiting User Credibility Information, Attention and Multi-task Learning
| Title | Rumor Detection by Exploiting User Credibility Information, Attention and Multi-task Learning |
| Authors | Quanzhi Li, Qiong Zhang, Luo Si |
| Abstract | In this study, we propose a new multi-task learning approach for rumor detection and stance classification tasks. This neural network model has a shared layer and two task specific layers. We incorporate the user credibility information into the rumor detection layer, and we also apply attention mechanism in the rumor detection process. The attended information include not only the hidden states in the rumor detection layer, but also the hidden states from the stance detection layer. The experiments on two datasets show that our proposed model outperforms the state-of-the-art rumor detection approaches. |
| Tasks | Multi-Task Learning, Stance Detection |
| Published | 2019-07-01 |
| URL | https://www.aclweb.org/anthology/P19-1113/ |
| https://www.aclweb.org/anthology/P19-1113 | |
| PWC | https://paperswithcode.com/paper/rumor-detection-by-exploiting-user |
| Repo | |
| Framework | |
Occlusion-Aware Networks for 3D Human Pose Estimation in Video
| Title | Occlusion-Aware Networks for 3D Human Pose Estimation in Video |
| Authors | Yu Cheng, Bo Yang, Bo Wang, Wending Yan, Robby T. Tan |
| Abstract | Occlusion is a key problem in 3D human pose estimation from a monocular video. To address this problem, we introduce an occlusion-aware deep-learning framework. By employing estimated 2D confidence heatmaps of keypoints and an optical-flow consistency constraint, we filter out the unreliable estimations of occluded keypoints. When occlusion occurs, we have incomplete 2D keypoints and feed them to our 2D and 3D temporal convolutional networks (2D and 3D TCNs) that enforce temporal smoothness to produce a complete 3D pose. By using incomplete 2D keypoints, instead of complete but incorrect ones, our networks are less affected by the error-prone estimations of occluded keypoints. Training the occlusion-aware 3D TCN requires pairs of a 3D pose and a 2D pose with occlusion labels. As no such a dataset is available, we introduce a “Cylinder Man Model” to approximate the occupation of body parts in 3D space. By projecting the model onto a 2D plane in different viewing angles, we obtain and label the occluded keypoints, providing us plenty of training data. In addition, we use this model to create a pose regularization constraint, preferring the 2D estimations of unreliable keypoints to be occluded. Our method outperforms state-of-the-art methods on Human 3.6M and HumanEva-I datasets. |
| Tasks | 3D Human Pose Estimation, Optical Flow Estimation, Pose Estimation |
| Published | 2019-10-01 |
| URL | http://openaccess.thecvf.com/content_ICCV_2019/html/Cheng_Occlusion-Aware_Networks_for_3D_Human_Pose_Estimation_in_Video_ICCV_2019_paper.html |
| http://openaccess.thecvf.com/content_ICCV_2019/papers/Cheng_Occlusion-Aware_Networks_for_3D_Human_Pose_Estimation_in_Video_ICCV_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/occlusion-aware-networks-for-3d-human-pose |
| Repo | |
| Framework | |
Improving Precision of Grammatical Error Correction with a Cheat Sheet
| Title | Improving Precision of Grammatical Error Correction with a Cheat Sheet |
| Authors | Mengyang Qiu, Xuejiao Chen, Maggie Liu, Krishna Parvathala, Apurva Patil, Jungyeul Park |
| Abstract | In this paper, we explore two approaches of generating error-focused phrases and examine whether these phrases can lead to better performance in grammatical error correction for the restricted track of BEA 2019 Shared Task on GEC. Our results show that phrases directly extracted from GEC corpora outperform phrases from statistical machine translation phrase table by a large margin. Appending error+context phrases to the original GEC corpora yields comparably high precision. We also explore the generation of artificial syntactic error sentences using error+context phrases for the unrestricted track. The additional training data greatly facilitates syntactic error correction (e.g., verb form) and contributes to better overall performance. |
| Tasks | Grammatical Error Correction, Machine Translation |
| Published | 2019-08-01 |
| URL | https://www.aclweb.org/anthology/W19-4425/ |
| https://www.aclweb.org/anthology/W19-4425 | |
| PWC | https://paperswithcode.com/paper/improving-precision-of-grammatical-error |
| Repo | |
| Framework | |
Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data
| Title | Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data |
| Authors | Roman Grundkiewicz, Marcin Junczys-Dowmunt, Kenneth Heafield |
| Abstract | Considerable effort has been made to address the data sparsity problem in neural grammatical error correction. In this work, we propose a simple and surprisingly effective unsupervised synthetic error generation method based on confusion sets extracted from a spellchecker to increase the amount of training data. Synthetic data is used to pre-train a Transformer sequence-to-sequence model, which not only improves over a strong baseline trained on authentic error-annotated data, but also enables the development of a practical GEC system in a scenario where little genuine error-annotated data is available. The developed systems placed first in the BEA19 shared task, achieving 69.47 and 64.24 F$_{0.5}$ in the restricted and low-resource tracks respectively, both on the W{&}I+LOCNESS test set. On the popular CoNLL 2014 test set, we report state-of-the-art results of 64.16 M{\mbox{$^2$}} for the submitted system, and 61.30 M{\mbox{$^2$}} for the constrained system trained on the NUCLE and Lang-8 data. |
| Tasks | Grammatical Error Correction |
| Published | 2019-08-01 |
| URL | https://www.aclweb.org/anthology/W19-4427/ |
| https://www.aclweb.org/anthology/W19-4427 | |
| PWC | https://paperswithcode.com/paper/neural-grammatical-error-correction-systems |
| Repo | |
| Framework | |
SkyScapes Fine-Grained Semantic Understanding of Aerial Scenes
| Title | SkyScapes Fine-Grained Semantic Understanding of Aerial Scenes |
| Authors | Seyed Majid Azimi, Corentin Henry, Lars Sommer, Arne Schumann, Eleonora Vig |
| Abstract | Understanding the complex urban infrastructure with centimeter-level accuracy is essential for many applications from autonomous driving to mapping, infrastructure monitoring, and urban management. Aerial images provide valuable information over a large area instantaneously; nevertheless, no current dataset captures the complexity of aerial scenes at the level of granularity required by real-world applications. To address this, we introduce SkyScapes, an aerial image dataset with highly-accurate, fine-grained annotations for pixel-level semantic labeling. SkyScapes provides annotations for 31 semantic categories ranging from large structures, such as buildings, roads and vegetation, to fine details, such as 12 (sub-)categories of lane markings. We have defined two main tasks on this dataset: dense semantic segmentation and multi-class lane-marking prediction. We carry out extensive experiments to evaluate state-of-the-art segmentation methods on SkyScapes. Existing methods struggle to deal with the wide range of classes, object sizes, scales, and fine details present. We therefore propose a novel multi-task model, which incorporates semantic edge detection and is better tuned for feature extraction from a wide range of scales. This model achieves notable improvements over the baselines in region outlines and level of detail on both tasks. |
| Tasks | Autonomous Driving, Edge Detection, Semantic Segmentation |
| Published | 2019-10-01 |
| URL | http://openaccess.thecvf.com/content_ICCV_2019/html/Azimi_SkyScapes__Fine-Grained_Semantic_Understanding_of_Aerial_Scenes_ICCV_2019_paper.html |
| http://openaccess.thecvf.com/content_ICCV_2019/papers/Azimi_SkyScapes__Fine-Grained_Semantic_Understanding_of_Aerial_Scenes_ICCV_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/skyscapes-fine-grained-semantic-understanding |
| Repo | |
| Framework | |
Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation
| Title | Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation |
| Authors | Fadi Biadsy, Ron J. Weiss, Pedro J. Moreno, Dimitri Kanvesky, Ye Jia |
| Abstract | We describe Parrotron, an end-to-end-trained speech-to-speech conversion model that maps an input spectrogram directly to another spectrogram, without utilizing any intermediate discrete representation. The network is composed of an encoder, spectrogram and phoneme decoders, followed by a vocoder to synthesize a time-domain waveform. We demonstrate that this model can be trained to normalize speech from any speaker regardless of accent, prosody, and background noise, into the voice of a single canonical target speaker with a fixed accent and consistent articulation and prosody. We further show that this normalization model can be adapted to normalize highly atypical speech from a deaf speaker, resulting in significant improvements in intelligibility and naturalness, measured via a speech recognizer and listening tests. Finally, demonstrating the utility of this model on other speech tasks, we show that the same model architecture can be trained to perform a speech separation task. |
| Tasks | Speech Enhancement |
| Published | 2019-07-12 |
| URL | https://arxiv.org/abs/1904.04169 |
| https://arxiv.org/pdf/1904.04169.pdf | |
| PWC | https://paperswithcode.com/paper/parrotron-an-end-to-end-speech-to-speech |
| Repo | |
| Framework | |
IFlyLegal: A Chinese Legal System for Consultation, Law Searching, and Document Analysis
| Title | IFlyLegal: A Chinese Legal System for Consultation, Law Searching, and Document Analysis |
| Authors | Ziyue Wang, Baoxin Wang, Xingyi Duan, Dayong Wu, Shijin Wang, Guoping Hu, Ting Liu |
| Abstract | Legal Tech is developed to help people with legal services and solve legal problems via machines. To achieve this, one of the key requirements for machines is to utilize legal knowledge and comprehend legal context. This can be fulfilled by natural language processing (NLP) techniques, for instance, text representation, text categorization, question answering (QA) and natural language inference, etc. To this end, we introduce a freely available Chinese Legal Tech system (IFlyLegal) that benefits from multiple NLP tasks. It is an integrated system that performs legal consulting, multi-way law searching, and legal document analysis by exploiting techniques such as deep contextual representations and various attention mechanisms. To our knowledge, IFlyLegal is the first Chinese legal system that employs up-to-date NLP techniques and caters for needs of different user groups, such as lawyers, judges, procurators, and clients. Since Jan, 2019, we have gathered 2,349 users and 28,238 page views (till June, 23, 2019). |
| Tasks | Natural Language Inference, Question Answering, Text Categorization |
| Published | 2019-11-01 |
| URL | https://www.aclweb.org/anthology/D19-3017/ |
| https://www.aclweb.org/anthology/D19-3017 | |
| PWC | https://paperswithcode.com/paper/iflylegal-a-chinese-legal-system-for |
| Repo | |
| Framework | |
Decomposed Local Models for Coordinate Structure Parsing
| Title | Decomposed Local Models for Coordinate Structure Parsing |
| Authors | Hiroki Teranishi, Hiroyuki Shindo, Yuji Matsumoto |
| Abstract | We propose a simple and accurate model for coordination boundary identification. Our model decomposes the task into three sub-tasks during training; finding a coordinator, identifying inside boundaries of a pair of conjuncts, and selecting outside boundaries of it. For inference, we make use of probabilities of coordinators and conjuncts in the CKY parsing to find the optimal combination of coordinate structures. Experimental results demonstrate that our model achieves state-of-the-art results, ensuring that the global structure of coordinations is consistent. |
| Tasks | |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/N19-1343/ |
| https://www.aclweb.org/anthology/N19-1343 | |
| PWC | https://paperswithcode.com/paper/decomposed-local-models-for-coordinate |
| Repo | |
| Framework | |
JU_ETCE_17_21 at SemEval-2019 Task 6: Efficient Machine Learning and Neural Network Approaches for Identifying and Categorizing Offensive Language in Tweets
| Title | JU_ETCE_17_21 at SemEval-2019 Task 6: Efficient Machine Learning and Neural Network Approaches for Identifying and Categorizing Offensive Language in Tweets |
| Authors | Preeti Mukherjee, Mainak Pal, Somnath Banerjee, Sudip Kumar Naskar |
| Abstract | This paper describes our system submissions as part of our participation (team name: JU{_}ETCE{_}17{_}21) in the SemEval 2019 shared task 6: {``}OffensEval: Identifying and Catego- rizing Offensive Language in Social Media{''}. We participated in all the three sub-tasks: i) Sub-task A: offensive language identification, ii) Sub-task B: automatic categorization of of- fense types, and iii) Sub-task C: offense target identification. We employed machine learn- ing as well as deep learning approaches for the sub-tasks. We employed Convolutional Neural Network (CNN) and Recursive Neu- ral Network (RNN) Long Short-Term Memory (LSTM) with pre-trained word embeddings. We used both word2vec and Glove pre-trained word embeddings. We obtained the best F1- score using CNN based model for sub-task A, LSTM based model for sub-task B and Lo- gistic Regression based model for sub-task C. Our best submissions achieved 0.7844, 0.5459 and 0.48 F1-scores for sub-task A, sub-task B and sub-task C respectively. | |
| Tasks | Language Identification, Word Embeddings |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2118/ |
| https://www.aclweb.org/anthology/S19-2118 | |
| PWC | https://paperswithcode.com/paper/ju_etce_17_21-at-semeval-2019-task-6 |
| Repo | |
| Framework | |
A seismic scaling relation for stellar age
| Title | A seismic scaling relation for stellar age |
| Authors | Earl Patrick Bellinger |
| Abstract | A simple solar scaling relation for estimating the ages of main-sequence stars from asteroseismic and spectroscopic data is developed. New seismic scaling relations for estimating mass and radius are presented as well, including a purely seismic radius scaling relation (i.e., no dependence on temperature). The relations show substantial improvement over the classical scaling relations and perform similarly well to grid-based modeling. |
| Tasks | |
| Published | 2019-03-11 |
| URL | https://arxiv.org/abs/1903.03110 |
| https://arxiv.org/pdf/1903.03110.pdf | |
| PWC | https://paperswithcode.com/paper/a-seismic-scaling-relation-for-stellar-age |
| Repo | |
| Framework | |
UWB@FinTOC-2019 Shared Task: Financial Document Title Detection
| Title | UWB@FinTOC-2019 Shared Task: Financial Document Title Detection |
| Authors | Pavel Kr{'a}l Tomas Hercig |
| Abstract | |
| Tasks | |
| Published | 2019-09-01 |
| URL | https://www.aclweb.org/anthology/W19-6411/ |
| https://www.aclweb.org/anthology/W19-6411 | |
| PWC | https://paperswithcode.com/paper/uwbfintoc-2019-shared-task-financial-document |
| Repo | |
| Framework | |
Convolutional Polar Codes on Channels with Memory using Tensor Networks
| Title | Convolutional Polar Codes on Channels with Memory using Tensor Networks |
| Authors | Benjamin Bourassa，Maxime Tremblay，David Poulin |
| Abstract | Arikan’s recursive code construction is designed to polarize a collection of memoryless channels into a set of good and a set of bad channels, and it can be efficiently decoded using successive cancellation [1]. It was recently shown that the same construction also polarizes channels with memory [2], and a generalization of successive cancellation decoder was proposed with a complexity that scales like the third power of the channel’s memory size [3]. In another line of work, the polar code construction was extended by replacing the block polarization kernel by a convoluted kernel [4]. Here, we present an efficient decoding algorithm for finite-state memory channels that can be applied to polar codes and convolutional polar codes. This generalization is most effectively described using the tensor network formalism, and the manuscript presents a self-contained description of the required basic concepts. We use numerical simulations to study the performance of these algorithms for practically relevant code sizes and find that the convolutional structure outperforms the standard polar codes on a variety of channels with memory. |
| Tasks | Tensor Networks |
| Published | 2019-06-15 |
| URL | https://arxiv.org/abs/1805.09378 |
| https://arxiv.org/abs/1805.09378 | |
| PWC | https://paperswithcode.com/paper/convolutional-polar-codes-on-channels-with |
| Repo | |
| Framework | |
