Paper Group ANR 18
Extracting Factual Min/Max Age Information from Clinical Trial Studies. Approximation of the objective insensitivity regions using Hierarchic Memetic Strategy coupled with Covariance Matrix Adaptation Evolutionary Strategy. Optimizing Speech Recognition For The Edge. Offline Writer Identification based on the Path Signature Feature. Bayesian Optimi …
Extracting Factual Min/Max Age Information from Clinical Trial Studies
| Title | Extracting Factual Min/Max Age Information from Clinical Trial Studies |
| Authors | Yufang Hou, Debasis Ganguly, Lea A. Deleris, Francesca Bonin |
| Abstract | Population age information is an essential characteristic of clinical trials. In this paper, we focus on extracting minimum and maximum (min/max) age values for the study samples from clinical research articles. Specifically, we investigate the use of a neural network model for question answering to address this information extraction task. The min/max age QA model is trained on the massive structured clinical study records from ClinicalTrials.gov. For each article, based on multiple min and max age values extracted from the QA model, we predict both actual min/max age values for the study samples and filter out non-factual age expressions. Our system improves the results over (i) a passage retrieval based IE system and (ii) a CRF-based system by a large margin when evaluated on an annotated dataset consisting of 50 research papers on smoking cessation. |
| Tasks | Question Answering |
| Published | 2019-04-05 |
| URL | http://arxiv.org/abs/1904.03262v1 |
| http://arxiv.org/pdf/1904.03262v1.pdf | |
| PWC | https://paperswithcode.com/paper/extracting-factual-minmax-age-information |
| Repo | |
| Framework | |
Approximation of the objective insensitivity regions using Hierarchic Memetic Strategy coupled with Covariance Matrix Adaptation Evolutionary Strategy
| Title | Approximation of the objective insensitivity regions using Hierarchic Memetic Strategy coupled with Covariance Matrix Adaptation Evolutionary Strategy |
| Authors | Jakub Sawicki, Maciej Smołka, Marcin Łoś, Robert Schaefer |
| Abstract | One of the most challenging types of ill-posedness in global optimization is the presence of insensitivity regions in design parameter space, so the identification of their shape will be crucial, if ill-posedness is irrecoverable. Such problems may be solved using global stochastic search followed by post-processing of a local sample and a local objective approximation. We propose a new approach of this type composed of Hierarchic Memetic Strategy (HMS) powered by the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) well-known as an effective, self-adaptable stochastic optimization algorithm and we leverage the distribution density knowledge it accumulates to better identify and separate insensitivity regions. The results of benchmarks prove that the improved HMS-CMA-ES strategy is effective in both the total computational cost and the accuracy of insensitivity region approximation. The reference data for the tests was obtained by means of a well-known effective strategy of multimodal stochastic optimization called the Niching Evolutionary Algorithm 2 (NEA2), that also uses CMA-ES as a component. |
| Tasks | Stochastic Optimization |
| Published | 2019-05-17 |
| URL | https://arxiv.org/abs/1905.07288v1 |
| https://arxiv.org/pdf/1905.07288v1.pdf | |
| PWC | https://paperswithcode.com/paper/approximation-of-the-objective-insensitivity |
| Repo | |
| Framework | |
Optimizing Speech Recognition For The Edge
| Title | Optimizing Speech Recognition For The Edge |
| Authors | Yuan Shangguan, Jian Li, Qiao Liang, Raziel Alvarez, Ian McGraw |
| Abstract | While most deployed speech recognition systems today still run on servers, we are in the midst of a transition towards deployments on edge devices. This leap to the edge is powered by the progression from traditional speech recognition pipelines to end-to-end (E2E) neural architectures, and the parallel development of more efficient neural network topologies and optimization techniques. Thus, we are now able to create highly accurate speech recognizers that are both small and fast enough to execute on typical mobile devices. In this paper, we begin with a baseline RNN-Transducer architecture comprised of Long Short-Term Memory (LSTM) layers. We then experiment with a variety of more computationally efficient layer types, as well as apply optimization techniques like neural connection pruning and parameter quantization to construct a small, high quality, on-device speech recognizer that is an order of magnitude smaller than the baseline system without any optimizations. |
| Tasks | Quantization, Speech Recognition |
| Published | 2019-09-26 |
| URL | https://arxiv.org/abs/1909.12408v3 |
| https://arxiv.org/pdf/1909.12408v3.pdf | |
| PWC | https://paperswithcode.com/paper/optimizing-speech-recognition-for-the-edge |
| Repo | |
| Framework | |
Offline Writer Identification based on the Path Signature Feature
| Title | Offline Writer Identification based on the Path Signature Feature |
| Authors | Songxuan Lai, Lianwen Jin |
| Abstract | In this paper, we propose a novel set of features for offline writer identification based on the path signature approach, which provides a principled way to express information contained in a path. By extracting local pathlets from handwriting contours, the path signature can also characterize the offline handwriting style. A codebook method based on the log path signature—a more compact way to express the path signature—is used in this work and shows competitive results on several benchmark offline writer identification datasets, namely the IAM, Firemaker, CVL and ICDAR2013 writer identification contest dataset. |
| Tasks | |
| Published | 2019-05-03 |
| URL | https://arxiv.org/abs/1905.01207v1 |
| https://arxiv.org/pdf/1905.01207v1.pdf | |
| PWC | https://paperswithcode.com/paper/offline-writer-identification-based-on-the |
| Repo | |
| Framework | |
Bayesian Optimization using Pseudo-Points
| Title | Bayesian Optimization using Pseudo-Points |
| Authors | Chao Qian, Hang Xiong, Ke Xue |
| Abstract | Bayesian optimization (BO) is a popular approach for expensive black-box optimization, with applications including parameter tuning, experimental design, robotics. BO usually models the objective function by a Gaussian process (GP), and iteratively samples the next data point by maximizing an acquisition function. In this paper, we propose a new general framework for BO by generating pseudo-points (i.e., data points whose objective values are not evaluated) to improve the GP model. With the classic acquisition function, i.e., upper confidence bound (UCB), we prove that the cumulative regret can be generally upper bounded. Experiments using UCB and other acquisition functions, i.e., probability of improvement (PI) and expectation of improvement (EI), on synthetic as well as real-world problems clearly show the advantage of generating pseudo-points. |
| Tasks | |
| Published | 2019-10-12 |
| URL | https://arxiv.org/abs/1910.05484v2 |
| https://arxiv.org/pdf/1910.05484v2.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-optimization-using-pseudo-points |
| Repo | |
| Framework | |
You Only Look & Listen Once: Towards Fast and Accurate Visual Grounding
| Title | You Only Look & Listen Once: Towards Fast and Accurate Visual Grounding |
| Authors | Chaorui Deng, Qi Wu, Guanghui Xu, Zhuliang Yu, Yanwu Xu, Kui Jia, Mingkui Tan |
| Abstract | Visual Grounding (VG) aims to locate the most relevant region in an image, based on a flexible natural language query but not a pre-defined label, thus it can be a more useful technique than object detection in practice. Most state-of-the-art methods in VG operate in a two-stage manner, wherein the first stage an object detector is adopted to generate a set of object proposals from the input image and the second stage is simply formulated as a cross-modal matching problem that finds the best match between the language query and all region proposals. This is rather inefficient because there might be hundreds of proposals produced in the first stage that need to be compared in the second stage, not to mention this strategy performs inaccurately. In this paper, we propose an simple, intuitive and much more elegant one-stage detection based method that joints the region proposal and matching stage as a single detection network. The detection is conditioned on the input query with a stack of novel Relation-to-Attention modules that transform the image-to-query relationship to an relation map, which is used to predict the bounding box directly without proposing large numbers of useless region proposals. During the inference, our approach is about 20x ~ 30x faster than previous methods and, remarkably, it achieves 18% ~ 41% absolute performance improvement on top of the state-of-the-art results on several benchmark datasets. We release our code and all the pre-trained models at https://github.com/openblack/rvg. |
| Tasks | Object Detection |
| Published | 2019-02-12 |
| URL | http://arxiv.org/abs/1902.04213v3 |
| http://arxiv.org/pdf/1902.04213v3.pdf | |
| PWC | https://paperswithcode.com/paper/you-only-look-listen-once-towards-fast-and |
| Repo | |
| Framework | |
CNAK : Cluster Number Assisted K-means
| Title | CNAK : Cluster Number Assisted K-means |
| Authors | Jayasree Saha, Jayanta Mukherjee |
| Abstract | Determining the number of clusters present in a dataset is an important problem in cluster analysis. Conventional clustering techniques generally assume this parameter to be provided up front. %user supplied. %Recently, robustness of any given clustering algorithm is analyzed to measure cluster stability/instability which in turn determines the cluster number. In this paper, we propose a method which analyzes cluster stability for predicting the cluster number. Under the same computational framework, the technique also finds representatives of the clusters. The method is apt for handling big data, as we design the algorithm using \emph{Monte-Carlo} simulation. Also, we explore a few pertinent issues found to be of also clustering. Experiments reveal that the proposed method is capable of identifying a single cluster. It is robust in handling high dimensional dataset and performs reasonably well over datasets having cluster imbalance. Moreover, it can indicate cluster hierarchy, if present. Overall we have observed significant improvement in speed and quality for predicting cluster numbers as well as the composition of clusters in a large dataset. |
| Tasks | |
| Published | 2019-11-20 |
| URL | https://arxiv.org/abs/1911.08871v1 |
| https://arxiv.org/pdf/1911.08871v1.pdf | |
| PWC | https://paperswithcode.com/paper/cnak-cluster-number-assisted-k-means |
| Repo | |
| Framework | |
Computational Register Analysis and Synthesis
| Title | Computational Register Analysis and Synthesis |
| Authors | Shlomo Engelson Argamon |
| Abstract | The study of register in computational language research has historically been divided into register analysis, seeking to determine the registerial character of a text or corpus, and register synthesis, seeking to generate a text in a desired register. This article surveys the different approaches to these disparate tasks. Register synthesis has tended to use more theoretically articulated notions of register and genre than analysis work, which often seeks to categorize on the basis of intuitive and somewhat incoherent notions of prelabeled ‘text types’. I argue that an integration of computational register analysis and synthesis will benefit register studies as a whole, by enabling a new large-scale research program in register studies. It will enable comprehensive global mapping of functional language varieties in multiple languages, including the relationships between them. Furthermore, computational methods together with high coverage systematically collected and analyzed data will thus enable rigorous empirical validation and refinement of different theories of register, which will have also implications for our understanding of linguistic variation in general. |
| Tasks | |
| Published | 2019-01-08 |
| URL | http://arxiv.org/abs/1901.02543v1 |
| http://arxiv.org/pdf/1901.02543v1.pdf | |
| PWC | https://paperswithcode.com/paper/computational-register-analysis-and-synthesis |
| Repo | |
| Framework | |
An Augmented Transformer Architecture for Natural Language Generation Tasks
| Title | An Augmented Transformer Architecture for Natural Language Generation Tasks |
| Authors | Hailiang Li, Adele Y. C. Wang, Yang Liu, Du Tang, Zhibin Lei, Wenye Li |
| Abstract | The Transformer based neural networks have been showing significant advantages on most evaluations of various natural language processing and other sequence-to-sequence tasks due to its inherent architecture based superiorities. Although the main architecture of the Transformer has been continuously being explored, little attention was paid to the positional encoding module. In this paper, we enhance the sinusoidal positional encoding algorithm by maximizing the variances between encoded consecutive positions to obtain additional promotion. Furthermore, we propose an augmented Transformer architecture encoded with additional linguistic knowledge, such as the Part-of-Speech (POS) tagging, to boost the performance on some natural language generation tasks, e.g., the automatic translation and summarization tasks. Experiments show that the proposed architecture attains constantly superior results compared to the vanilla Transformer. |
| Tasks | Part-Of-Speech Tagging, Text Generation |
| Published | 2019-10-30 |
| URL | https://arxiv.org/abs/1910.13634v1 |
| https://arxiv.org/pdf/1910.13634v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-augmented-transformer-architecture-for |
| Repo | |
| Framework | |
Identifying Experts in Software Libraries and Frameworks among GitHub Users
| Title | Identifying Experts in Software Libraries and Frameworks among GitHub Users |
| Authors | Joao Eduardo Montandon, Luciana Lourdes Silva, Marco Tulio Valente |
| Abstract | Software development increasingly depends on libraries and frameworks to increase productivity and reduce time-to-market. Despite this fact, we still lack techniques to assess developers expertise in widely popular libraries and frameworks. In this paper, we evaluate the performance of unsupervised (based on clustering) and supervised machine learning classifiers (Random Forest and SVM) to identify experts in three popular JavaScript libraries: facebook/react, mongodb/node-mongodb, and socketio/socket.io. First, we collect 13 features about developers activity on GitHub projects, including commits on source code files that depend on these libraries. We also build a ground truth including the expertise of 575 developers on the studied libraries, as self-reported by them in a survey. Based on our findings, we document the challenges of using machine learning classifiers to predict expertise in software libraries, using features extracted from GitHub. Then, we propose a method to identify library experts based on clustering feature data from GitHub; by triangulating the results of this method with information available on Linkedin profiles, we show that it is able to recommend dozens of GitHub users with evidences of being experts in the studied JavaScript libraries. We also provide a public dataset with the expertise of 575 developers on the studied libraries. |
| Tasks | |
| Published | 2019-03-19 |
| URL | http://arxiv.org/abs/1903.08113v1 |
| http://arxiv.org/pdf/1903.08113v1.pdf | |
| PWC | https://paperswithcode.com/paper/identifying-experts-in-software-libraries-and |
| Repo | |
| Framework | |
Part of speech tagging for code switched data
| Title | Part of speech tagging for code switched data |
| Authors | Fahad AlGhamdi, Giovanni Molina, Mona Diab, Thamar Solorio, Abdelati Hawwari, Victor Soto, Julia Hirschberg |
| Abstract | We address the problem of Part of Speech tagging (POS) in the context of linguistic code switching (CS). CS is the phenomenon where a speaker switches between two languages or variants of the same language within or across utterances, known as intra-sentential or inter-sentential CS, respectively. Processing CS data is especially challenging in intra-sentential data given state of the art monolingual NLP technology since such technology is geared toward the processing of one language at a time. In this paper we explore multiple strategies of applying state of the art POS taggers to CS data. We investigate the landscape in two CS language pairs, Spanish-English and Modern Standard Arabic-Arabic dialects. We compare the use of two POS taggers vs. a unified tagger trained on CS data. Our results show that applying a machine learning framework using two state of the art POS taggers achieves better performance compared to all other approaches that we investigate. |
| Tasks | Part-Of-Speech Tagging |
| Published | 2019-09-28 |
| URL | https://arxiv.org/abs/1909.13006v2 |
| https://arxiv.org/pdf/1909.13006v2.pdf | |
| PWC | https://paperswithcode.com/paper/part-of-speech-tagging-for-code-switched-data-1 |
| Repo | |
| Framework | |
Optimal transport mapping via input convex neural networks
| Title | Optimal transport mapping via input convex neural networks |
| Authors | Ashok Vardhan Makkuva, Amirhossein Taghvaei, Sewoong Oh, Jason D. Lee |
| Abstract | In this paper, we present a novel and principled approach to learn the optimal transport between two distributions, from samples. Guided by the optimal transport theory, we learn the optimal Kantorovich potential which induces the optimal transport map. This involves learning two convex functions, by solving a novel minimax optimization. Building upon recent advances in the field of input convex neural networks, we propose a new framework where the gradient of one convex function represents the optimal transport mapping. Numerical experiments confirm that we learn the optimal transport mapping. This approach ensures that the transport mapping we find is optimal independent of how we initialize the neural networks. Further, target distributions from a discontinuous support can be easily captured, as gradient of a convex function naturally models a {\em discontinuous} transport mapping. |
| Tasks | |
| Published | 2019-08-28 |
| URL | https://arxiv.org/abs/1908.10962v1 |
| https://arxiv.org/pdf/1908.10962v1.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-transport-mapping-via-input-convex |
| Repo | |
| Framework | |
Exploring ways to incorporate additional knowledge to improve Natural Language Commonsense Question Answering
| Title | Exploring ways to incorporate additional knowledge to improve Natural Language Commonsense Question Answering |
| Authors | Arindam Mitra, Pratyay Banerjee, Kuntal Kumar Pal, Swaroop Mishra, Chitta Baral |
| Abstract | DARPA and Allen AI have proposed a collection of datasets to encourage research in Question Answering domains where (commonsense) knowledge is expected to play an important role. Recent language models such as BERT and GPT that have been pre-trained on Wikipedia articles and books, have shown decent performance with little fine-tuning on several such Multiple Choice Question-Answering (MCQ) datasets. Our goal in this work is to develop methods to incorporate additional (commonsense) knowledge into language model based approaches for better question answering in such domains. In this work we first identify external knowledge sources, and show that the performance further improves when a set of facts retrieved through IR is prepended to each MCQ question during both training and test phase. We then explore if the performance can be further improved by providing task specific knowledge in different manners or by employing different strategies for using the available knowledge. We present three different modes of passing knowledge and five different models of using knowledge including the standard BERT MCQ model. We also propose a novel architecture to deal with situations where information to answer the MCQ question is scattered over multiple knowledge sentences. We take 200 predictions from each of our best models and analyze how often the given knowledge is useful, how many times the given knowledge is useful but system failed to use it and some other metrices to see the scope of further improvements. |
| Tasks | Language Modelling, Question Answering |
| Published | 2019-09-19 |
| URL | https://arxiv.org/abs/1909.08855v1 |
| https://arxiv.org/pdf/1909.08855v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-ways-to-incorporate-additional |
| Repo | |
| Framework | |
Hierarchical model-based policy optimization: from actions to action sequences and back
| Title | Hierarchical model-based policy optimization: from actions to action sequences and back |
| Authors | Daniel McNamee |
| Abstract | We develop a normative framework for hierarchical model-based policy optimization based on applying second-order methods in the space of all possible state-action paths. The resulting natural path gradient performs policy updates in a manner which is sensitive to the long-range correlational structure of the induced stationary state-action densities. We demonstrate that the natural path gradient can be computed exactly given an environment dynamics model and depends on expressions akin to higher-order successor representations. In simulation, we show that the priorization of local policy updates in the resulting policy flow indeed reflects the intuitive state-space hierarchy in several toy problems. |
| Tasks | |
| Published | 2019-11-28 |
| URL | https://arxiv.org/abs/1912.01448v2 |
| https://arxiv.org/pdf/1912.01448v2.pdf | |
| PWC | https://paperswithcode.com/paper/hierarchical-model-based-policy-optimization |
| Repo | |
| Framework | |
Backpropagation-Friendly Eigendecomposition
| Title | Backpropagation-Friendly Eigendecomposition |
| Authors | Wei Wang, Zheng Dang, Yinlin Hu, Pascal Fua, Mathieu Salzmann |
| Abstract | Eigendecomposition (ED) is widely used in deep networks. However, the backpropagation of its results tends to be numerically unstable, whether using ED directly or approximating it with the Power Iteration method, particularly when dealing with large matrices. While this can be mitigated by partitioning the data in small and arbitrary groups, doing so has no theoretical basis and makes its impossible to exploit the power of ED to the full. In this paper, we introduce a numerically stable and differentiable approach to leveraging eigenvectors in deep networks. It can handle large matrices without requiring to split them. We demonstrate the better robustness of our approach over standard ED and PI for ZCA whitening, an alternative to batch normalization, and for PCA denoising, which we introduce as a new normalization strategy for deep networks, aiming to further denoise the network’s features. |
| Tasks | Denoising |
| Published | 2019-06-21 |
| URL | https://arxiv.org/abs/1906.09023v2 |
| https://arxiv.org/pdf/1906.09023v2.pdf | |
| PWC | https://paperswithcode.com/paper/backpropagation-friendly-eigendecomposition |
| Repo | |
| Framework | |
