Paper Group NANR 104
Skip-Gram − Zipf + Uniform = Vector Additivity. Cross-Spectral Factor Analysis. Maxing and Ranking with Few Assumptions. Integration Methods and Optimization Algorithms. Generating Natural Answers by Incorporating Copying and Retrieving Mechanisms in Sequence-to-Sequence Learning. Machine Translation and Automated Analysis of the Sumerian Languag …
Skip-Gram − Zipf + Uniform = Vector Additivity
| Title | Skip-Gram − Zipf + Uniform = Vector Additivity |
| Authors | Alex Gittens, Dimitris Achlioptas, Michael W. Mahoney |
| Abstract | In recent years word-embedding models have gained great popularity due to their remarkable performance on several tasks, including word analogy questions and caption generation. An unexpected {}side-effect{''} of such models is that their vectors often exhibit compositionality, i.e., \textit{adding}two word-vectors results in a vector that is only a small angle away from the vector of a word representing the semantic composite of the original words, e.g., {}man{''} + {}royal{''} = {}king{''}. This work provides a theoretical justification for the presence of additive compositionality in word vectors learned using the Skip-Gram model. In particular, it shows that additive compositionality holds in an even stricter sense (small distance rather than small angle) under certain assumptions on the process generating the corpus. As a corollary, it explains the success of vector calculus in solving word analogies. When these assumptions do not hold, this work describes the correct non-linear composition operator. Finally, this work establishes a connection between the Skip-Gram model and the Sufficient Dimensionality Reduction (SDR) framework of Globerson and Tishby: the parameters of SDR models can be obtained from those of Skip-Gram models simply by adding information on symbol frequencies. This shows that Skip-Gram embeddings are optimal in the sense of Globerson and Tishby and, further, implies that the heuristics commonly used to approximately fit Skip-Gram models can be used to fit SDR models. |
| Tasks | Dimensionality Reduction, Word Embeddings |
| Published | 2017-07-01 |
| URL | https://www.aclweb.org/anthology/P17-1007/ |
| https://www.aclweb.org/anthology/P17-1007 | |
| PWC | https://paperswithcode.com/paper/skip-gram-zipf-uniform-vector-additivity |
| Repo | |
| Framework | |
Cross-Spectral Factor Analysis
| Title | Cross-Spectral Factor Analysis |
| Authors | Neil Gallagher, Kyle R. Ulrich, Austin Talbot, Kafui Dzirasa, Lawrence Carin, David E. Carlson |
| Abstract | In neuropsychiatric disorders such as schizophrenia or depression, there is often a disruption in the way that regions of the brain synchronize with one another. To facilitate understanding of network-level synchronization between brain regions, we introduce a novel model of multisite low-frequency neural recordings, such as local field potentials (LFPs) and electroencephalograms (EEGs). The proposed model, named Cross-Spectral Factor Analysis (CSFA), breaks the observed signal into factors defined by unique spatio-spectral properties. These properties are granted to the factors via a Gaussian process formulation in a multiple kernel learning framework. In this way, the LFP signals can be mapped to a lower dimensional space in a way that retains information of relevance to neuroscientists. Critically, the factors are interpretable. The proposed approach empirically allows similar performance in classifying mouse genotype and behavioral context when compared to commonly used approaches that lack the interpretability of CSFA. We also introduce a semi-supervised approach, termed discriminative CSFA (dCSFA). CSFA and dCSFA provide useful tools for understanding neural dynamics, particularly by aiding in the design of causal follow-up experiments. |
| Tasks | |
| Published | 2017-12-01 |
| URL | http://papers.nips.cc/paper/7260-cross-spectral-factor-analysis |
| http://papers.nips.cc/paper/7260-cross-spectral-factor-analysis.pdf | |
| PWC | https://paperswithcode.com/paper/cross-spectral-factor-analysis |
| Repo | |
| Framework | |
Maxing and Ranking with Few Assumptions
| Title | Maxing and Ranking with Few Assumptions |
| Authors | Moein Falahatgar, Yi Hao, Alon Orlitsky, Venkatadheeraj Pichapati, Vaishakh Ravindrakumar |
| Abstract | PAC maximum selection (maxing) and ranking of $n$ elements via random pairwise comparisons have diverse applications and have been studied under many models and assumptions. With just one simple natural assumption: strong stochastic transitivity, we show that maxing can be performed with linearly many comparisons yet ranking requires quadratically many. With no assumptions at all, we show that for the Borda-score metric, maximum selection can be performed with linearly many comparisons and ranking can be performed with $\mathcal{O}(n\log n)$ comparisons. |
| Tasks | |
| Published | 2017-12-01 |
| URL | http://papers.nips.cc/paper/7281-maxing-and-ranking-with-few-assumptions |
| http://papers.nips.cc/paper/7281-maxing-and-ranking-with-few-assumptions.pdf | |
| PWC | https://paperswithcode.com/paper/maxing-and-ranking-with-few-assumptions |
| Repo | |
| Framework | |
Integration Methods and Optimization Algorithms
| Title | Integration Methods and Optimization Algorithms |
| Authors | Damien Scieur, Vincent Roulet, Francis Bach, Alexandre D’Aspremont |
| Abstract | We show that accelerated optimization methods can be seen as particular instances of multi-step integration schemes from numerical analysis, applied to the gradient flow equation. Compared with recent advances in this vein, the differential equation considered here is the basic gradient flow, and we derive a class of multi-step schemes which includes accelerated algorithms, using classical conditions from numerical analysis. Multi-step schemes integrate the differential equation using larger step sizes, which intuitively explains the acceleration phenomenon. |
| Tasks | |
| Published | 2017-12-01 |
| URL | http://papers.nips.cc/paper/6711-integration-methods-and-optimization-algorithms |
| http://papers.nips.cc/paper/6711-integration-methods-and-optimization-algorithms.pdf | |
| PWC | https://paperswithcode.com/paper/integration-methods-and-optimization |
| Repo | |
| Framework | |
Generating Natural Answers by Incorporating Copying and Retrieving Mechanisms in Sequence-to-Sequence Learning
| Title | Generating Natural Answers by Incorporating Copying and Retrieving Mechanisms in Sequence-to-Sequence Learning |
| Authors | Shizhu He, Cao Liu, Kang Liu, Jun Zhao |
| Abstract | Generating answer with natural language sentence is very important in real-world question answering systems, which needs to obtain a right answer as well as a coherent natural response. In this paper, we propose an end-to-end question answering system called COREQA in sequence-to-sequence learning, which incorporates copying and retrieving mechanisms to generate natural answers within an encoder-decoder framework. Specifically, in COREQA, the semantic units (words, phrases and entities) in a natural answer are dynamically predicted from the vocabulary, copied from the given question and/or retrieved from the corresponding knowledge base jointly. Our empirical study on both synthetic and real-world datasets demonstrates the efficiency of COREQA, which is able to generate correct, coherent and natural answers for knowledge inquired questions. |
| Tasks | Question Answering |
| Published | 2017-07-01 |
| URL | https://www.aclweb.org/anthology/P17-1019/ |
| https://www.aclweb.org/anthology/P17-1019 | |
| PWC | https://paperswithcode.com/paper/generating-natural-answers-by-incorporating |
| Repo | |
| Framework | |
Machine Translation and Automated Analysis of the Sumerian Language
| Title | Machine Translation and Automated Analysis of the Sumerian Language |
| Authors | {'E}milie Pag{'e}-Perron, Maria Sukhareva, Ilya Khait, Christian Chiarcos |
| Abstract | This paper presents a newly funded international project for machine translation and automated analysis of ancient cuneiform languages where NLP specialists and Assyriologists collaborate to create an information retrieval system for Sumerian. This research is conceived in response to the need to translate large numbers of administrative texts that are only available in transcription, in order to make them accessible to a wider audience. The methodology includes creation of a specialized NLP pipeline and also the use of linguistic linked open data to increase access to the results. |
| Tasks | Information Retrieval, Machine Translation |
| Published | 2017-08-01 |
| URL | https://www.aclweb.org/anthology/W17-2202/ |
| https://www.aclweb.org/anthology/W17-2202 | |
| PWC | https://paperswithcode.com/paper/machine-translation-and-automated-analysis-of |
| Repo | |
| Framework | |
Using Neural Word Embeddings in the Analysis of the Clinical Semantic Verbal Fluency Task
| Title | Using Neural Word Embeddings in the Analysis of the Clinical Semantic Verbal Fluency Task |
| Authors | Nicklas Linz, Johannes Tr{"o}ger, Alex, Jan ersson, Alex K{"o}nig, ra |
| Abstract | |
| Tasks | Word Embeddings |
| Published | 2017-01-01 |
| URL | https://www.aclweb.org/anthology/W17-6926/ |
| https://www.aclweb.org/anthology/W17-6926 | |
| PWC | https://paperswithcode.com/paper/using-neural-word-embeddings-in-the-analysis |
| Repo | |
| Framework | |
Semantic Parsing of Pre-university Math Problems
| Title | Semantic Parsing of Pre-university Math Problems |
| Authors | Takuya Matsuzaki, Takumi Ito, Hidenao Iwane, Hirokazu Anai, Noriko H. Arai |
| Abstract | We have been developing an end-to-end math problem solving system that accepts natural language input. The current paper focuses on how we analyze the problem sentences to produce logical forms. We chose a hybrid approach combining a shallow syntactic analyzer and a manually-developed lexicalized grammar. A feature of the grammar is that it is extensively typed on the basis of a formal ontology for pre-university math. These types are helpful in semantic disambiguation inside and across sentences. Experimental results show that the hybrid system produces a well-formed logical form with 88{%} precision and 56{%} recall. |
| Tasks | Semantic Parsing |
| Published | 2017-07-01 |
| URL | https://www.aclweb.org/anthology/P17-1195/ |
| https://www.aclweb.org/anthology/P17-1195 | |
| PWC | https://paperswithcode.com/paper/semantic-parsing-of-pre-university-math |
| Repo | |
| Framework | |
An End-to-End Model for Question Answering over Knowledge Base with Cross-Attention Combining Global Knowledge
| Title | An End-to-End Model for Question Answering over Knowledge Base with Cross-Attention Combining Global Knowledge |
| Authors | Yanchao Hao, Yuanzhe Zhang, Kang Liu, Shizhu He, Zhanyi Liu, Hua Wu, Jun Zhao |
| Abstract | With the rapid growth of knowledge bases (KBs) on the web, how to take full advantage of them becomes increasingly important. Question answering over knowledge base (KB-QA) is one of the promising approaches to access the substantial knowledge. Meanwhile, as the neural network-based (NN-based) methods develop, NN-based KB-QA has already achieved impressive results. However, previous work did not put more emphasis on question representation, and the question is converted into a fixed vector regardless of its candidate answers. This simple representation strategy is not easy to express the proper information in the question. Hence, we present an end-to-end neural network model to represent the questions and their corresponding scores dynamically according to the various candidate answer aspects via cross-attention mechanism. In addition, we leverage the global knowledge inside the underlying KB, aiming at integrating the rich KB information into the representation of the answers. As a result, it could alleviates the out-of-vocabulary (OOV) problem, which helps the cross-attention model to represent the question more precisely. The experimental results on WebQuestions demonstrate the effectiveness of the proposed approach. |
| Tasks | Information Retrieval, Question Answering, Semantic Parsing |
| Published | 2017-07-01 |
| URL | https://www.aclweb.org/anthology/P17-1021/ |
| https://www.aclweb.org/anthology/P17-1021 | |
| PWC | https://paperswithcode.com/paper/an-end-to-end-model-for-question-answering |
| Repo | |
| Framework | |
ZikaHack 2016: A digital disease detection competition
| Title | ZikaHack 2016: A digital disease detection competition |
| Authors | Dillon C Adam, Jitendra Jonnagaddala, Daniel Han-Chen, Sean Batongbacal, Luan Almeida, Jing Z Zhu, Jenny J Yang, Jumail M Mundekkat, Steven Badman, Abrar Chughtai, C Raina MacIntyre |
| Abstract | Effective response to infectious diseases outbreaks relies on the rapid and early detection of those outbreaks. Invalidated, yet timely and openly available digital information can be used for the early detection of outbreaks. Public health surveillance authorities can exploit these early warnings to plan and co-ordinate rapid surveillance and emergency response programs. In 2016, a digital disease detection competition named ZikaHack was launched. The objective of the competition was for multidisciplinary teams to design, develop and demonstrate innovative digital disease detection solutions to retrospectively detect the 2015-16 Brazilian Zika virus outbreak earlier than traditional surveillance methods. In this paper, an overview of the ZikaHack competition is provided. The challenges and lessons learned in organizing this competition are also discussed for use by other researchers interested in organizing similar competitions. |
| Tasks | |
| Published | 2017-11-01 |
| URL | https://www.aclweb.org/anthology/W17-5806/ |
| https://www.aclweb.org/anthology/W17-5806 | |
| PWC | https://paperswithcode.com/paper/zikahack-2016-a-digital-disease-detection |
| Repo | |
| Framework | |
Hierarchy Through Composition with Multitask LMDPs
| Title | Hierarchy Through Composition with Multitask LMDPs |
| Authors | Andrew M. Saxe, Adam C. Earle, Benjamin Rosman |
| Abstract | Hierarchical architectures are critical to the scalability of reinforcement learning methods. Most current hierarchical frameworks execute actions serially, with macro-actions comprising sequences of primitive actions. We propose a novel alternative to these control hierarchies based on concurrent execution of many actions in parallel. Our scheme exploits the guaranteed concurrent compositionality provided by the linearly solvable Markov decision process (LMDP) framework, which naturally enables a learning agent to draw on several macro-actions simultaneously to solve new tasks. We introduce the Multitask LMDP module, which maintains a parallel distributed representation of tasks and may be stacked to form deep hierarchies abstracted in space and time. |
| Tasks | |
| Published | 2017-08-01 |
| URL | https://icml.cc/Conferences/2017/Schedule?showEvent=725 |
| http://proceedings.mlr.press/v70/saxe17a/saxe17a.pdf | |
| PWC | https://paperswithcode.com/paper/hierarchy-through-composition-with-multitask |
| Repo | |
| Framework | |
HLP@UPenn at SemEval-2017 Task 4A: A simple, self-optimizing text classification system combining dense and sparse vectors
| Title | HLP@UPenn at SemEval-2017 Task 4A: A simple, self-optimizing text classification system combining dense and sparse vectors |
| Authors | Abeed Sarker, Graciela Gonzalez |
| Abstract | We present a simple supervised text classification system that combines sparse and dense vector representations of words, and generalized representations of words via clusters. The sparse vectors are generated from word n-gram sequences (1-3). The dense vector representations of words (embeddings) are learned by training a neural network to predict neighboring words in a large unlabeled dataset. To classify a text segment, the different representations of it are concatenated, and the classification is performed using Support Vector Machines (SVM). Our system is particularly intended for use by non-experts of natural language processing and machine learning, and, therefore, the system does not require any manual tuning of parameters or weights. Given a training set, the system automatically generates the training vectors, optimizes the relevant hyper-parameters for the SVM classifier, and trains the classification model. We evaluated this system on the SemEval-2017 English sentiment analysis task. In terms of average F1-score, our system obtained 8th position out of 39 submissions (F1-score: 0.632, average recall: 0.637, accuracy: 0.646). |
| Tasks | Epidemiology, Sentiment Analysis, Text Classification |
| Published | 2017-08-01 |
| URL | https://www.aclweb.org/anthology/S17-2105/ |
| https://www.aclweb.org/anthology/S17-2105 | |
| PWC | https://paperswithcode.com/paper/hlpupenn-at-semeval-2017-task-4a-a-simple |
| Repo | |
| Framework | |
Book Review: Automatic Detection of Verbal Deception by Eileen Fitzpatrick, Joan Bachenko and Tommaso Fornaciari
| Title | Book Review: Automatic Detection of Verbal Deception by Eileen Fitzpatrick, Joan Bachenko and Tommaso Fornaciari |
| Authors | Yoong Keok Lee |
| Abstract | |
| Tasks | |
| Published | 2017-04-01 |
| URL | https://www.aclweb.org/anthology/J17-1008/ |
| https://www.aclweb.org/anthology/J17-1008 | |
| PWC | https://paperswithcode.com/paper/book-review-automatic-detection-of-verbal |
| Repo | |
| Framework | |
Generating and Evaluating Summaries for Partial Email Threads: Conversational Bayesian Surprise and Silver Standards
| Title | Generating and Evaluating Summaries for Partial Email Threads: Conversational Bayesian Surprise and Silver Standards |
| Authors | Jordon Johnson, Vaden Masrani, Giuseppe Carenini, Raymond Ng |
| Abstract | We define and motivate the problem of summarizing partial email threads. This problem introduces the challenge of generating reference summaries for partial threads when human annotation is only available for the threads as a whole, particularly when the human-selected sentences are not uniformly distributed within the threads. We propose an oracular algorithm for generating these reference summaries with arbitrary length, and we are making the resulting dataset publicly available. In addition, we apply a recent unsupervised method based on Bayesian Surprise that incorporates background knowledge into partial thread summarization, extend it with conversational features, and modify the mechanism by which it handles redundancy. Experiments with our method indicate improved performance over the baseline for shorter partial threads; and our results suggest that the potential benefits of background knowledge to partial thread summarization should be further investigated with larger datasets. |
| Tasks | |
| Published | 2017-08-01 |
| URL | https://www.aclweb.org/anthology/W17-5532/ |
| https://www.aclweb.org/anthology/W17-5532 | |
| PWC | https://paperswithcode.com/paper/generating-and-evaluating-summaries-for |
| Repo | |
| Framework | |
Learning Translations via Matrix Completion
| Title | Learning Translations via Matrix Completion |
| Authors | Derry Tanti Wijaya, Brendan Callahan, John Hewitt, Jie Gao, Xiao Ling, Marianna Apidianaki, Chris Callison-Burch |
| Abstract | Bilingual Lexicon Induction is the task of learning word translations without bilingual parallel corpora. We model this task as a matrix completion problem, and present an effective and extendable framework for completing the matrix. This method harnesses diverse bilingual and monolingual signals, each of which may be incomplete or noisy. Our model achieves state-of-the-art performance for both high and low resource languages. |
| Tasks | Machine Translation, Matrix Completion, Word Embeddings |
| Published | 2017-09-01 |
| URL | https://www.aclweb.org/anthology/D17-1152/ |
| https://www.aclweb.org/anthology/D17-1152 | |
| PWC | https://paperswithcode.com/paper/learning-translations-via-matrix-completion |
| Repo | |
| Framework | |
