Paper Group NANR 254
Keynote: Setting up a Machine Translation Program for IARPA. Simulating Language Evolution: a Tool for Historical Linguistics. Contextual Combinatorial Multi-armed Bandits with Volatile Arms and Submodular Reward. Diverse Ensemble Evolution: Curriculum Data-Model Marriage. DCFEE: A Document-level Chinese Financial Event Extraction System based on A …
Keynote: Setting up a Machine Translation Program for IARPA
| Title | Keynote: Setting up a Machine Translation Program for IARPA |
| Authors | Carl Rubino |
| Abstract | |
| Tasks | Machine Translation |
| Published | 2018-03-01 |
| URL | https://www.aclweb.org/anthology/W18-1902/ |
| https://www.aclweb.org/anthology/W18-1902 | |
| PWC | https://paperswithcode.com/paper/keynote-setting-up-a-machine-translation |
| Repo | |
| Framework | |
Simulating Language Evolution: a Tool for Historical Linguistics
| Title | Simulating Language Evolution: a Tool for Historical Linguistics |
| Authors | Alina Maria Ciobanu, Liviu P. Dinu |
| Abstract | Language change across space and time is one of the main concerns in historical linguistics. In this paper, we develop a language evolution simulator: a web-based tool for word form production to assist in historical linguistics, in studying the evolution of the languages. Given a word in a source language, the system automatically predicts how the word evolves in a target language. The method that we propose is language-agnostic and does not use any external knowledge, except for the training word pairs. |
| Tasks | Machine Translation |
| Published | 2018-08-01 |
| URL | https://www.aclweb.org/anthology/C18-2015/ |
| https://www.aclweb.org/anthology/C18-2015 | |
| PWC | https://paperswithcode.com/paper/simulating-language-evolution-a-tool-for |
| Repo | |
| Framework | |
Contextual Combinatorial Multi-armed Bandits with Volatile Arms and Submodular Reward
| Title | Contextual Combinatorial Multi-armed Bandits with Volatile Arms and Submodular Reward |
| Authors | Lixing Chen, Jie Xu, Zhuo Lu |
| Abstract | In this paper, we study the stochastic contextual combinatorial multi-armed bandit (CC-MAB) framework that is tailored for volatile arms and submodular reward functions. CC-MAB inherits properties from both contextual bandit and combinatorial bandit: it aims to select a set of arms in each round based on the side information (a.k.a. context) associated with the arms. By volatile arms'', we mean that the available arms to select from in each round may change; and by submodular rewards’', we mean that the total reward achieved by selected arms is not a simple sum of individual rewards but demonstrates a feature of diminishing returns determined by the relations between selected arms (e.g. relevance and redundancy). Volatile arms and submodular rewards are often seen in many real-world applications, e.g. recommender systems and crowdsourcing, in which multi-armed bandit (MAB) based strategies are extensively applied. Although there exist works that investigate these issues separately based on standard MAB, jointly considering all these issues in a single MAB problem requires very different algorithm design and regret analysis. Our algorithm CC-MAB provides an online decision-making policy in a contextual and combinatorial bandit setting and effectively addresses the issues raised by volatile arms and submodular reward functions. The proposed algorithm is proved to achieve $O(cT^{\frac{2\alpha+D}{3\alpha + D}}\log(T))$ regret after a span of $T$ rounds. The performance of CC-MAB is evaluated by experiments conducted on a real-world crowdsourcing dataset, and the result shows that our algorithm outperforms the prior art. |
| Tasks | Decision Making, Multi-Armed Bandits, Recommendation Systems |
| Published | 2018-12-01 |
| URL | http://papers.nips.cc/paper/7586-contextual-combinatorial-multi-armed-bandits-with-volatile-arms-and-submodular-reward |
| http://papers.nips.cc/paper/7586-contextual-combinatorial-multi-armed-bandits-with-volatile-arms-and-submodular-reward.pdf | |
| PWC | https://paperswithcode.com/paper/contextual-combinatorial-multi-armed-bandits |
| Repo | |
| Framework | |
Diverse Ensemble Evolution: Curriculum Data-Model Marriage
| Title | Diverse Ensemble Evolution: Curriculum Data-Model Marriage |
| Authors | Tianyi Zhou, Shengjie Wang, Jeff A. Bilmes |
| Abstract | We study a new method (``Diverse Ensemble Evolution (DivE$^2$)'') to train an ensemble of machine learning models that assigns data to models at each training epoch based on each model’s current expertise and an intra- and inter-model diversity reward. DivE$^2$ schedules, over the course of training epochs, the relative importance of these characteristics; it starts by selecting easy samples for each model, and then gradually adjusts towards the models having specialized and complementary expertise on subsets of the training data, thereby encouraging high accuracy of the ensemble. We utilize an intra-model diversity term on data assigned to each model, and an inter-model diversity term on data assigned to pairs of models, to penalize both within-model and cross-model redundancy. We formulate the data-model marriage problem as a generalized bipartite matching, represented as submodular maximization subject to two matroid constraints. DivE$^2$ solves a sequence of continuous-combinatorial optimizations with slowly varying objectives and constraints. The combinatorial part handles the data-model marriage while the continuous part updates model parameters based on the assignments. In experiments, DivE$^2$ outperforms other ensemble training methods under a variety of model aggregation techniques, while also maintaining competitive efficiency. | |
| Tasks | |
| Published | 2018-12-01 |
| URL | http://papers.nips.cc/paper/7831-diverse-ensemble-evolution-curriculum-data-model-marriage |
| http://papers.nips.cc/paper/7831-diverse-ensemble-evolution-curriculum-data-model-marriage.pdf | |
| PWC | https://paperswithcode.com/paper/diverse-ensemble-evolution-curriculum-data |
| Repo | |
| Framework | |
DCFEE: A Document-level Chinese Financial Event Extraction System based on Automatically Labeled Training Data
| Title | DCFEE: A Document-level Chinese Financial Event Extraction System based on Automatically Labeled Training Data |
| Authors | Hang Yang, Yubo Chen, Kang Liu, Yang Xiao, Jun Zhao |
| Abstract | We present an event extraction framework to detect event mentions and extract events from the document-level financial news. Up to now, methods based on supervised learning paradigm gain the highest performance in public datasets (such as ACE2005, KBP2015). These methods heavily depend on the manually labeled training data. However, in particular areas, such as financial, medical and judicial domains, there is no enough labeled data due to the high cost of data labeling process. Moreover, most of the current methods focus on extracting events from one sentence, but an event is usually expressed by multiple sentences in one document. To solve these problems, we propose a Document-level Chinese Financial Event Extraction (DCFEE) system which can automatically generate a large scaled labeled data and extract events from the whole document. Experimental results demonstrate the effectiveness of it |
| Tasks | |
| Published | 2018-07-01 |
| URL | https://www.aclweb.org/anthology/P18-4009/ |
| https://www.aclweb.org/anthology/P18-4009 | |
| PWC | https://paperswithcode.com/paper/dcfee-a-document-level-chinese-financial |
| Repo | |
| Framework | |
Towards Automation of Sense-type Identification of Verbs in OntoSenseNet
| Title | Towards Automation of Sense-type Identification of Verbs in OntoSenseNet |
| Authors | Sreekavitha Parupalli, Vijjini Anvesh Rao, Radhika Mamidi |
| Abstract | In this paper, we discuss the enrichment of a manually developed resource, OntoSenseNet for Telugu. OntoSenseNet is a sense annotated resource that marks each verb of Telugu with a primary and a secondary sense. The area of research is relatively recent but has a large scope of development. We provide an introductory work to enrich the OntoSenseNet to promote further research in Telugu. Classifiers are adopted to learn the sense relevant features of the words in the resource and also to automate the tagging of sense-types for verbs. We perform a comparative analysis of different classifiers applied on OntoSenseNet. The results of the experiment prove that automated enrichment of the resource is effective using SVM classifiers and Adaboost ensemble. |
| Tasks | |
| Published | 2018-07-01 |
| URL | https://www.aclweb.org/anthology/W18-3511/ |
| https://www.aclweb.org/anthology/W18-3511 | |
| PWC | https://paperswithcode.com/paper/towards-automation-of-sense-type-1 |
| Repo | |
| Framework | |
Regular Expression Guided Entity Mention Mining from Noisy Web Data
| Title | Regular Expression Guided Entity Mention Mining from Noisy Web Data |
| Authors | Shanshan Zhang, Lihong He, Slobodan Vucetic, Eduard Dragut |
| Abstract | Many important entity types in web documents, such as dates, times, email addresses, and course numbers, follow or closely resemble patterns that can be described by Regular Expressions (REs). Due to a vast diversity of web documents and ways in which they are being generated, even seemingly straightforward tasks such as identifying mentions of date in a document become very challenging. It is reasonable to claim that it is impossible to create a RE that is capable of identifying such entities from web documents with perfect precision and recall. Rather than abandoning REs as a go-to approach for entity detection, this paper explores ways to combine the expressive power of REs, ability of deep learning to learn from large data, and human-in-the loop approach into a new integrated framework for entity identification from web data. The framework starts by creating or collecting the existing REs for a particular type of an entity. Those REs are then used over a large document corpus to collect weak labels for the entity mentions and a neural network is trained to predict those RE-generated weak labels. Finally, a human expert is asked to label a small set of documents and the neural network is fine tuned on those documents. The experimental evaluation on several entity identification problems shows that the proposed framework achieves impressive accuracy, while requiring very modest human effort. |
| Tasks | Named Entity Recognition, Text Classification |
| Published | 2018-10-01 |
| URL | https://www.aclweb.org/anthology/D18-1224/ |
| https://www.aclweb.org/anthology/D18-1224 | |
| PWC | https://paperswithcode.com/paper/regular-expression-guided-entity-mention |
| Repo | |
| Framework | |
Approximation Guarantees for Adaptive Sampling
| Title | Approximation Guarantees for Adaptive Sampling |
| Authors | Eric Balkanski, Yaron Singer |
| Abstract | In this paper we analyze an adaptive sampling approach for submodular maximization. Adaptive sampling is a technique that has recently been shown to achieve a constant factor approximation guarantee for submodular maximization under a cardinality constraint with exponentially fewer adaptive rounds than any previously studied constant factor approximation algorithm for this problem. Adaptivity quantifies the number of sequential rounds that an algorithm makes when function evaluations can be executed in parallel and is the parallel running time of an algorithm, up to low order terms. Adaptive sampling achieves its exponential speedup at the expense of approximation. In theory, it is guaranteed to produce a solution that is a 1/3 approximation to the optimum. Nevertheless, experiments show that adaptive sampling techniques achieve far better values in practice. In this paper we provide theoretical justification for this phenomenon. In particular, we show that under very mild conditions of curvature of a function, adaptive sampling techniques achieve an approximation arbitrarily close to 1/2 while maintaining their low adaptivity. Furthermore, we show that the approximation ratio approaches 1 in direct relationship to a homogeneity property of the submodular function. In addition, we conduct experiments on real data sets in which the curvature and homogeneity properties can be easily manipulated and demonstrate the relationship between approximation and curvature, as well as the effectiveness of adaptive sampling in practice. |
| Tasks | |
| Published | 2018-07-01 |
| URL | https://icml.cc/Conferences/2018/Schedule?showEvent=1882 |
| http://proceedings.mlr.press/v80/balkanski18a/balkanski18a.pdf | |
| PWC | https://paperswithcode.com/paper/approximation-guarantees-for-adaptive |
| Repo | |
| Framework | |
Surface Realization Shared Task 2018 (SR18): The Tilburg University Approach
| Title | Surface Realization Shared Task 2018 (SR18): The Tilburg University Approach |
| Authors | Thiago Castro Ferreira, S Wubben, er, Emiel Krahmer |
| Abstract | This study describes the approach developed by the Tilburg University team to the shallow task of the Multilingual Surface Realization Shared Task 2018 (SR18). Based on (Castro Ferreira et al., 2017), the approach works by first preprocessing an input dependency tree into an ordered linearized string, which is then realized using a statistical machine translation model. Our approach shows promising results, with BLEU scores above 50 for 5 different languages (English, French, Italian, Portuguese and Spanish) and above 35 for the Dutch language. |
| Tasks | Machine Translation |
| Published | 2018-07-01 |
| URL | https://www.aclweb.org/anthology/W18-3604/ |
| https://www.aclweb.org/anthology/W18-3604 | |
| PWC | https://paperswithcode.com/paper/surface-realization-shared-task-2018-sr18-the |
| Repo | |
| Framework | |
Proceedings of the Third Conference on Machine Translation: Research Papers
| Title | Proceedings of the Third Conference on Machine Translation: Research Papers |
| Authors | |
| Abstract | |
| Tasks | Machine Translation |
| Published | 2018-10-01 |
| URL | https://www.aclweb.org/anthology/W18-6300/ |
| https://www.aclweb.org/anthology/W18-6300 | |
| PWC | https://paperswithcode.com/paper/proceedings-of-the-third-conference-on |
| Repo | |
| Framework | |
Learning to Coordinate with Coordination Graphs in Repeated Single-Stage Multi-Agent Decision Problems
| Title | Learning to Coordinate with Coordination Graphs in Repeated Single-Stage Multi-Agent Decision Problems |
| Authors | Eugenio Bargiacchi, Timothy Verstraeten, Diederik Roijers, Ann Nowé, Hado Hasselt |
| Abstract | Learning to coordinate between multiple agents is an important problem in many reinforcement learning problems. Key to learning to coordinate is exploiting loose couplings, i.e., conditional independences between agents. In this paper we study learning in repeated fully cooperative games, multi-agent multi-armed bandits (MAMABs), in which the expected rewards can be expressed as a coordination graph. We propose multi-agent upper confidence exploration (MAUCE), a new algorithm for MAMABs that exploits loose couplings, which enables us to prove a regret bound that is logarithmic in the number of arm pulls and only linear in the number of agents. We empirically compare MAUCE to sparse cooperative Q-learning, and a state-of-the-art combinatorial bandit approach, and show that it performs much better on a variety of settings, including learning control policies for wind farms. |
| Tasks | Multi-Armed Bandits, Q-Learning |
| Published | 2018-07-01 |
| URL | https://icml.cc/Conferences/2018/Schedule?showEvent=2177 |
| http://proceedings.mlr.press/v80/bargiacchi18a/bargiacchi18a.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-coordinate-with-coordination |
| Repo | |
| Framework | |
Power-law efficient neural codes provide general link between perceptual bias and discriminability
| Title | Power-law efficient neural codes provide general link between perceptual bias and discriminability |
| Authors | Michael Morais, Jonathan W. Pillow |
| Abstract | Recent work in theoretical neuroscience has shown that information-theoretic “efficient” neural codes, which allocate neural resources to maximize the mutual information between stimuli and neural responses, give rise to a lawful relationship between perceptual bias and discriminability that is observed across a wide variety of psychophysical tasks in human observers (Wei & Stocker 2017). Here we generalize these results to show that the same law arises under a much larger family of optimal neural codes, introducing a unifying framework that we call power-law efficient coding. Specifically, we show that the same lawful relationship between bias and discriminability arises whenever Fisher information is allocated proportional to any power of the prior distribution. This family includes neural codes that are optimal for minimizing Lp error for any p, indicating that the lawful relationship observed in human psychophysical data does not require information-theoretically optimal neural codes. Furthermore, we derive the exact constant of proportionality governing the relationship between bias and discriminability for different power laws (which includes information-theoretically optimal codes, where the power is 2, and so-called discrimax codes, where power is 1/2), and different choices of optimal decoder. As a bonus, our framework provides new insights into “anti-Bayesian” perceptual biases, in which percepts are biased away from the center of mass of the prior. We derive an explicit formula that clarifies precisely which combinations of neural encoder and decoder can give rise to such biases. |
| Tasks | |
| Published | 2018-12-01 |
| URL | http://papers.nips.cc/paper/7754-power-law-efficient-neural-codes-provide-general-link-between-perceptual-bias-and-discriminability |
| http://papers.nips.cc/paper/7754-power-law-efficient-neural-codes-provide-general-link-between-perceptual-bias-and-discriminability.pdf | |
| PWC | https://paperswithcode.com/paper/power-law-efficient-neural-codes-provide |
| Repo | |
| Framework | |
Learning Text Representations for 500K Classification Tasks on Named Entity Disambiguation
| Title | Learning Text Representations for 500K Classification Tasks on Named Entity Disambiguation |
| Authors | Ander Barrena, Aitor Soroa, Eneko Agirre |
| Abstract | Named Entity Disambiguation algorithms typically learn a single model for all target entities. In this paper we present a word expert model and train separate deep learning models for each target entity string, yielding 500K classification tasks. This gives us the opportunity to benchmark popular text representation alternatives on this massive dataset. In order to face scarce training data we propose a simple data-augmentation technique and transfer-learning. We show that bag-of-word-embeddings are better than LSTMs for tasks with scarce training data, while the situation is reversed when having larger amounts. Transferring a LSTM which is learned on all datasets is the most effective context representation option for the word experts in all frequency bands. The experiments show that our system trained on out-of-domain Wikipedia data surpass comparable NED systems which have been trained on in-domain training data. |
| Tasks | Data Augmentation, Entity Disambiguation, Entity Linking, Entity Resolution, Transfer Learning, Word Embeddings |
| Published | 2018-10-01 |
| URL | https://www.aclweb.org/anthology/K18-1017/ |
| https://www.aclweb.org/anthology/K18-1017 | |
| PWC | https://paperswithcode.com/paper/learning-text-representations-for-500k |
| Repo | |
| Framework | |
Compressing Neural Networks using the Variational Information Bottelneck
| Title | Compressing Neural Networks using the Variational Information Bottelneck |
| Authors | Bin Dai, Chen Zhu, Baining Guo, David Wipf |
| Abstract | Neural networks can be compressed to reduce memory and computational requirements, or to increase accuracy by facilitating the use of a larger base architecture. In this paper we focus on pruning individual neurons, which can simultaneously trim model size, FLOPs, and run-time memory. To improve upon the performance of existing compression algorithms we utilize the information bottleneck principle instantiated via a tractable variational bound. Minimization of this information theoretic bound reduces the redundancy between adjacent layers by aggregating useful information into a subset of neurons that can be preserved. In contrast, the activations of disposable neurons are shut off via an attractive form of sparse regularization that emerges naturally from this framework, providing tangible advantages over traditional sparsity penalties without contributing additional tuning parameters to the energy landscape. We demonstrate state-of-the-art compression rates across an array of datasets and network architectures. |
| Tasks | |
| Published | 2018-07-01 |
| URL | https://icml.cc/Conferences/2018/Schedule?showEvent=2411 |
| http://proceedings.mlr.press/v80/dai18d/dai18d.pdf | |
| PWC | https://paperswithcode.com/paper/compressing-neural-networks-using-the |
| Repo | |
| Framework | |
Anytime Neural Network: a Versatile Trade-off Between Computation and Accuracy
| Title | Anytime Neural Network: a Versatile Trade-off Between Computation and Accuracy |
| Authors | Hanzhang Hu, Debadeepta Dey, Martial Hebert, J. Andrew Bagnell |
| Abstract | We present an approach for anytime predictions in deep neural networks (DNNs). For each test sample, an anytime predictor produces a coarse result quickly, and then continues to refine it until the test-time computational budget is depleted. Such predictors can address the growing computational problem of DNNs by automatically adjusting to varying test-time budgets. In this work, we study a \emph{general} augmentation to feed-forward networks to form anytime neural networks (ANNs) via auxiliary predictions and losses. Specifically, we point out a blind-spot in recent studies in such ANNs: the importance of high final accuracy. In fact, we show on multiple recognition data-sets and architectures that by having near-optimal final predictions in small anytime models, we can effectively double the speed of large ones to reach corresponding accuracy level. We achieve such speed-up with simple weighting of anytime losses that oscillate during training. We also assemble a sequence of exponentially deepening ANNs, to achieve both theoretically and practically near-optimal anytime results at any budget, at the cost of a constant fraction of additional consumed budget. |
| Tasks | |
| Published | 2018-01-01 |
| URL | https://openreview.net/forum?id=SJa1Nk10b |
| https://openreview.net/pdf?id=SJa1Nk10b | |
| PWC | https://paperswithcode.com/paper/anytime-neural-network-a-versatile-trade-off |
| Repo | |
| Framework | |
