February 1, 2020

3149 words 15 mins read

Paper Group AWR 141

Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension. Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms. CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth. Business Taxonomy Construction Using Concept-Level Hierarchical Clustering. Cascade RPN: Delv …

Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension


Title	Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension
Authors	Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li
Abstract	This paper considers the reading comprehension task in which multiple documents are given as input. Prior work has shown that a pipeline of retriever, reader, and reranker can improve the overall performance. However, the pipeline system is inefficient since the input is re-encoded within each module, and is unable to leverage upstream components to help downstream training. In this work, we present RE$^3$QA, a unified question answering model that combines context retrieving, reading comprehension, and answer reranking to predict the final answer. Unlike previous pipelined approaches, RE$^3$QA shares contextualized text representation across different components, and is carefully designed to use high-quality upstream outputs (e.g., retrieved context or candidate answers) for directly supervising downstream modules (e.g., the reader or the reranker). As a result, the whole network can be trained end-to-end to avoid the context inconsistency problem. Experiments show that our model outperforms the pipelined baseline and achieves state-of-the-art results on two versions of TriviaQA and two variants of SQuAD.
Tasks	Question Answering, Reading Comprehension
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04618v1
PDF	https://arxiv.org/pdf/1906.04618v1.pdf
PWC	https://paperswithcode.com/paper/retrieve-read-rerank-towards-end-to-end-multi
Repo	https://github.com/huminghao16/RE3QA
Framework	pytorch

Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms


Title	Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms
Authors	Mahesh Chandra Mukkamala, Peter Ochs
Abstract	Matrix Factorization is a popular non-convex optimization problem, for which alternating minimization schemes are mostly used. They usually suffer from the major drawback that the solution is biased towards one of the optimization variables. A remedy is non-alternating schemes. However, due to a lack of Lipschitz continuity of the gradient in matrix factorization problems, convergence cannot be guaranteed. A recently developed approach relies on the concept of Bregman distances, which generalizes the standard Euclidean distance. We exploit this theory by proposing a novel Bregman distance for matrix factorization problems, which, at the same time, allows for simple/closed form update steps. Therefore, for non-alternating schemes, such as the recently introduced Bregman Proximal Gradient (BPG) method and an inertial variant Convex–Concave Inertial BPG (CoCaIn BPG), convergence of the whole sequence to a stationary point is proved for Matrix Factorization. In several experiments, we observe a superior performance of our non-alternating schemes in terms of speed and objective value at the limit point.
Tasks
Published	2019-05-22
URL	https://arxiv.org/abs/1905.09050v2
PDF	https://arxiv.org/pdf/1905.09050v2.pdf
PWC	https://paperswithcode.com/paper/beyond-alternating-updates-for-matrix
Repo	https://github.com/mmahesh/cocain-bpg-matrix-factorization
Framework	none

CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth


Title	CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth
Authors	Jose M. Facil, Benjamin Ummenhofer, Huizhong Zhou, Luis Montesano, Thomas Brox, Javier Civera
Abstract	Single-view depth estimation suffers from the problem that a network trained on images from one camera does not generalize to images taken with a different camera model. Thus, changing the camera model requires collecting an entirely new training dataset. In this work, we propose a new type of convolution that can take the camera parameters into account, thus allowing neural networks to learn calibration-aware patterns. Experiments confirm that this improves the generalization capabilities of depth prediction networks considerably, and clearly outperforms the state of the art when the train and test images are acquired with different cameras.
Tasks	3D Depth Estimation, Calibration, Depth Estimation, Domain Generalization
Published	2019-04-03
URL	http://arxiv.org/abs/1904.02028v1
PDF	http://arxiv.org/pdf/1904.02028v1.pdf
PWC	https://paperswithcode.com/paper/cam-convs-camera-aware-multi-scale
Repo	https://github.com/jmfacil/camconvs
Framework	tf

Business Taxonomy Construction Using Concept-Level Hierarchical Clustering


Title	Business Taxonomy Construction Using Concept-Level Hierarchical Clustering
Authors	Haodong Bai, Frank Z. Xing, Erik Cambria, Win-Bin Huang
Abstract	Business taxonomies are indispensable tools for investors to do equity research and make professional decisions. However, to identify the structure of industry sectors in an emerging market is challenging for two reasons. First, existing taxonomies are designed for mature markets, which may not be the appropriate classification for small companies with innovative business models. Second, emerging markets are fast-developing, thus the static business taxonomies cannot promptly reflect the new features. In this article, we propose a new method to construct business taxonomies automatically from the content of corporate annual reports. Extracted concepts are hierarchically clustered using greedy affinity propagation. Our method requires less supervision and is able to discover new terms. Experiments and evaluation on the Chinese National Equities Exchange and Quotations (NEEQ) market show several advantages of the business taxonomy we build. Our results provide an effective tool for understanding and investing in the new growth companies.
Tasks	Business Taxonomy Construction
Published	2019-06-24
URL	https://arxiv.org/abs/1906.09694v1
PDF	https://arxiv.org/pdf/1906.09694v1.pdf
PWC	https://paperswithcode.com/paper/business-taxonomy-construction-using-concept
Repo	https://github.com/SenticNet/neeq-annual-reports
Framework	none

Cascade RPN: Delving into High-Quality Region Proposal Network with Adaptive Convolution


Title	Cascade RPN: Delving into High-Quality Region Proposal Network with Adaptive Convolution
Authors	Thang Vu, Hyunjun Jang, Trung X. Pham, Chang D. Yoo
Abstract	This paper considers an architecture referred to as Cascade Region Proposal Network (Cascade RPN) for improving the region-proposal quality and detection performance by \textit{systematically} addressing the limitation of the conventional RPN that \textit{heuristically defines} the anchors and \textit{aligns} the features to the anchors. First, instead of using multiple anchors with predefined scales and aspect ratios, Cascade RPN relies on a \textit{single anchor} per location and performs multi-stage refinement. Each stage is progressively more stringent in defining positive samples by starting out with an anchor-free metric followed by anchor-based metrics in the ensuing stages. Second, to attain alignment between the features and the anchors throughout the stages, \textit{adaptive convolution} is proposed that takes the anchors in addition to the image features as its input and learns the sampled features guided by the anchors. A simple implementation of a two-stage Cascade RPN achieves AR 13.4 points higher than that of the conventional RPN, surpassing any existing region proposal methods. When adopting to Fast R-CNN and Faster R-CNN, Cascade RPN can improve the detection mAP by 3.1 and 3.5 points, respectively. The code is made publicly available at \url{https://github.com/thangvubk/Cascade-RPN.git}.
Tasks	Object Detection
Published	2019-09-15
URL	https://arxiv.org/abs/1909.06720v2
PDF	https://arxiv.org/pdf/1909.06720v2.pdf
PWC	https://paperswithcode.com/paper/cascade-rpn-delving-into-high-quality-region
Repo	https://github.com/thangvubk/Cascade-RPN
Framework	pytorch

Extended 2D Consensus Hippocampus Segmentation


Title	Extended 2D Consensus Hippocampus Segmentation
Authors	Diedre Carmo, Bruna Silva, Clarissa Yasuda, Letícia Rittner, Roberto Lotufo
Abstract	Hippocampus segmentation plays a key role in diagnosing various brain disorders such as Alzheimer’s disease, epilepsy, multiple sclerosis, cancer, depression and others. Nowadays, segmentation is still mainly performed manually by specialists. Segmentation done by experts is considered to be a gold-standard when evaluating automated methods, buts it is a time consuming and arduos task, requiring specialized personnel. In recent years, efforts have been made to achieve reliable automated segmentation. For years the best performing authomatic methods were multi atlas based with around 80-85% Dice coefficient and very time consuming, but machine learning methods are recently rising with promising time and accuracy performance. A method for volumetric hippocampus segmentation is presented, based on the consensus of tri-planar U-Net inspired fully convolutional networks (FCNNs), with some modifications, including residual connections, VGG weight transfers, batch normalization and a patch extraction technique employing data from neighbor patches. A study on the impact of our modifications to the classical U-Net architecture was performed. Our method achieves cutting edge performance in our dataset, with around 96% volumetric Dice accuracy in our test data. In a public validation dataset, HARP, we achieve 87.48% DICE. GPU execution time is in the order of seconds per volume, and source code is publicly available. Also, masks are shown to be similar to other recent state-of-the-art hippocampus segmentation methods in a third dataset, without manual annotations.
Tasks
Published	2019-02-12
URL	https://arxiv.org/abs/1902.04487v4
PDF	https://arxiv.org/pdf/1902.04487v4.pdf
PWC	https://paperswithcode.com/paper/extended-2d-volumetric-consensus-hippocampus
Repo	https://github.com/dscarmo/E2DConsHipseg
Framework	pytorch

The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction


Title	The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction
Authors	Dimitrios Alikaniotis, Vipul Raheja
Abstract	Recent work on Grammatical Error Correction (GEC) has highlighted the importance of language modeling in that it is certainly possible to achieve good performance by comparing the probabilities of the proposed edits. At the same time, advancements in language modeling have managed to generate linguistic output, which is almost indistinguishable from that of human-generated text. In this paper, we up the ante by exploring the potential of more sophisticated language models in GEC and offer some key insights on their strengths and weaknesses. We show that, in line with recent results in other NLP tasks, Transformer architectures achieve consistently high performance and provide a competitive baseline for future machine learning models.
Tasks	Grammatical Error Correction, Language Modelling
Published	2019-06-04
URL	https://arxiv.org/abs/1906.01733v1
PDF	https://arxiv.org/pdf/1906.01733v1.pdf
PWC	https://paperswithcode.com/paper/the-unreasonable-effectiveness-of-transformer
Repo	https://github.com/todd-cook/ML-You-Can-Use/blob/master/probablistic_language_modeling/automatic_grammatical_error_corrections_using_BERT_GPT2.ipynb
Framework	none

Measuring Bias in Contextualized Word Representations


Title	Measuring Bias in Contextualized Word Representations
Authors	Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, Yulia Tsvetkov
Abstract	Contextual word embeddings such as BERT have achieved state of the art performance in numerous NLP tasks. Since they are optimized to capture the statistical properties of training data, they tend to pick up on and amplify social stereotypes present in the data as well. In this study, we (1)~propose a template-based method to quantify bias in BERT; (2)~show that this method obtains more consistent results in capturing social biases than the traditional cosine based method; and (3)~conduct a case study, evaluating gender bias in a downstream task of Gender Pronoun Resolution. Although our case study focuses on gender bias, the proposed technique is generalizable to unveiling other biases, including in multiclass settings, such as racial and religious biases.
Tasks	Word Embeddings
Published	2019-06-18
URL	https://arxiv.org/abs/1906.07337v1
PDF	https://arxiv.org/pdf/1906.07337v1.pdf
PWC	https://paperswithcode.com/paper/measuring-bias-in-contextualized-word
Repo	https://github.com/MLforHealth/HurtfulWords
Framework	pytorch

A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains


Title	A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains
Authors	Lyndon Chan, Mahdi S. Hosseini, Konstantinos N. Plataniotis
Abstract	Recently proposed methods for weakly-supervised semantic segmentation have achieved impressive performance in predicting pixel classes despite being trained with only image labels which lack positional information. Because image annotations are cheaper and quicker to generate, weak supervision is more feasible for training segmentation algorithms in certain datasets. These methods have been predominantly developed on natural scene images and it is unclear whether they can be simply transferred to other domains with different characteristics, such as histopathology and satellite images, and still perform well. Little work has been conducted in the literature on applying weakly-supervised methods to these other image domains; it is unknown how to determine whether certain methods are more suitable for certain datasets, and how to determine the best method to use for a new dataset. This paper evaluates state-of-the-art weakly-supervised semantic segmentation methods on natural scene, histopathology, and satellite image datasets. We also analyze the compatibility of the methods for each dataset and present some principles for applying weakly-supervised semantic segmentation on an unseen image dataset.
Tasks	Semantic Segmentation, Weakly-Supervised Semantic Segmentation
Published	2019-12-24
URL	https://arxiv.org/abs/1912.11186v1
PDF	https://arxiv.org/pdf/1912.11186v1.pdf
PWC	https://paperswithcode.com/paper/a-comprehensive-analysis-of-weakly-supervised
Repo	https://github.com/lyndonchan/wsss-analysis
Framework	tf

Attentive Multi-Task Deep Reinforcement Learning


Title	Attentive Multi-Task Deep Reinforcement Learning
Authors	Timo Bram, Gino Brunner, Oliver Richter, Roger Wattenhofer
Abstract	Sharing knowledge between tasks is vital for efficient learning in a multi-task setting. However, most research so far has focused on the easier case where knowledge transfer is not harmful, i.e., where knowledge from one task cannot negatively impact the performance on another task. In contrast, we present an approach to multi-task deep reinforcement learning based on attention that does not require any a-priori assumptions about the relationships between tasks. Our attention network automatically groups task knowledge into sub-networks on a state level granularity. It thereby achieves positive knowledge transfer if possible, and avoids negative transfer in cases where tasks interfere. We test our algorithm against two state-of-the-art multi-task/transfer learning approaches and show comparable or superior performance while requiring fewer network parameters.
Tasks	Transfer Learning
Published	2019-07-05
URL	https://arxiv.org/abs/1907.02874v1
PDF	https://arxiv.org/pdf/1907.02874v1.pdf
PWC	https://paperswithcode.com/paper/attentive-multi-task-deep-reinforcement
Repo	https://github.com/braemt/attentive-multi-task-deep-reinforcement-learning
Framework	tf

Optimal Estimation of Generalized Average Treatment Effects using Kernel Optimal Matching


Title	Optimal Estimation of Generalized Average Treatment Effects using Kernel Optimal Matching
Authors	Nathan Kallus, Michele Santacatterina
Abstract	In causal inference, a variety of causal effect estimands have been studied, including the sample, uncensored, target, conditional, optimal subpopulation, and optimal weighted average treatment effects. Ad-hoc methods have been developed for each estimand based on inverse probability weighting (IPW) and on outcome regression modeling, but these may be sensitive to model misspecification, practical violations of positivity, or both. The contribution of this paper is twofold. First, we formulate the generalized average treatment effect (GATE) to unify these causal estimands as well as their IPW estimates. Second, we develop a method based on Kernel Optimal Matching (KOM) to optimally estimate GATE and to find the GATE most easily estimable by KOM, which we term the Kernel Optimal Weighted Average Treatment Effect. KOM provides uniform control on the conditional mean squared error of a weighted estimator over a class of models while simultaneously controlling for precision. We study its theoretical properties and evaluate its comparative performance in a simulation study. We illustrate the use of KOM for GATE estimation in two case studies: comparing spine surgical interventions and studying the effect of peer support on people living with HIV.
Tasks	Causal Inference
Published	2019-08-13
URL	https://arxiv.org/abs/1908.04748v2
PDF	https://arxiv.org/pdf/1908.04748v2.pdf
PWC	https://paperswithcode.com/paper/optimal-estimation-of-generalized-average
Repo	https://github.com/michelesantacatterina/KOM
Framework	none

RLBench: The Robot Learning Benchmark & Learning Environment


Title	RLBench: The Robot Learning Benchmark & Learning Environment
Authors	Stephen James, Zicong Ma, David Rovick Arrojo, Andrew J. Davison
Abstract	We present a challenging new benchmark and learning-environment for robot learning: RLBench. The benchmark features 100 completely unique, hand-designed tasks ranging in difficulty, from simple target reaching and door opening, to longer multi-stage tasks, such as opening an oven and placing a tray in it. We provide an array of both proprioceptive observations and visual observations, which include rgb, depth, and segmentation masks from an over-the-shoulder stereo camera and an eye-in-hand monocular camera. Uniquely, each task comes with an infinite supply of demos through the use of motion planners operating on a series of waypoints given during task creation time; enabling an exciting flurry of demonstration-based learning. RLBench has been designed with scalability in mind; new tasks, along with their motion-planned demos, can be easily created and then verified by a series of tools, allowing users to submit their own tasks to the RLBench task repository. This large-scale benchmark aims to accelerate progress in a number of vision-guided manipulation research areas, including: reinforcement learning, imitation learning, multi-task learning, geometric computer vision, and in particular, few-shot learning. With the benchmark’s breadth of tasks and demonstrations, we propose the first large-scale few-shot challenge in robotics. We hope that the scale and diversity of RLBench offers unparalleled research opportunities in the robot learning community and beyond.
Tasks	Few-Shot Learning, Imitation Learning, Multi-Task Learning
Published	2019-09-26
URL	https://arxiv.org/abs/1909.12271v1
PDF	https://arxiv.org/pdf/1909.12271v1.pdf
PWC	https://paperswithcode.com/paper/rlbench-the-robot-learning-benchmark-learning
Repo	https://github.com/stepjam/PyRep
Framework	none

ALiPy: Active Learning in Python


Title	ALiPy: Active Learning in Python
Authors	Ying-Peng Tang, Guo-Xiang Li, Sheng-Jun Huang
Abstract	Supervised machine learning methods usually require a large set of labeled examples for model training. However, in many real applications, there are plentiful unlabeled data but limited labeled data; and the acquisition of labels is costly. Active learning (AL) reduces the labeling cost by iteratively selecting the most valuable data to query their labels from the annotator. This article introduces a Python toobox ALiPy for active learning. ALiPy provides a module based implementation of active learning framework, which allows users to conveniently evaluate, compare and analyze the performance of active learning methods. In the toolbox, multiple options are available for each component of the learning framework, including data process, active selection, label query, results visualization, etc. In addition to the implementations of more than 20 state-of-the-art active learning algorithms, ALiPy also supports users to easily configure and implement their own approaches under different active learning settings, such as AL for multi-label data, AL with noisy annotators, AL with different costs and so on. The toolbox is well-documented and open-source on Github, and can be easily installed through PyPI.
Tasks	Active Learning
Published	2019-01-12
URL	http://arxiv.org/abs/1901.03802v1
PDF	http://arxiv.org/pdf/1901.03802v1.pdf
PWC	https://paperswithcode.com/paper/alipy-active-learning-in-python
Repo	https://github.com/micka-charpak/ProjetAnalyse
Framework	none

Categorical Co-Frequency Analysis: Clustering Diagnosis Codes to Predict Hospital Readmissions


Title	Categorical Co-Frequency Analysis: Clustering Diagnosis Codes to Predict Hospital Readmissions
Authors	Hallee E. Wong, Brianna C. Heggeseth, Steven J. Miller
Abstract	Accurately predicting patients’ risk of 30-day hospital readmission would enable hospitals to efficiently allocate resource-intensive interventions. We develop a new method, Categorical Co-Frequency Analysis (CoFA), for clustering diagnosis codes from the International Classification of Diseases (ICD) according to the similarity in relationships between covariates and readmission risk. CoFA measures the similarity between diagnoses by the frequency with which two diagnoses are split in the same direction versus split apart in random forests to predict readmission risk. Applying CoFA to de-identified data from Berkshire Medical Center, we identified three groups of diagnoses that vary in readmission risk. To evaluate CoFA, we compared readmission risk models using ICD majors and CoFA groups to a baseline model without diagnosis variables. We found substituting ICD majors for the CoFA-identified clusters simplified the model without compromising the accuracy of predictions. Fitting separate models for each ICD major and CoFA group did not improve predictions, suggesting that readmission risk may be more homogeneous that heterogeneous across diagnosis groups.
Tasks
Published	2019-09-01
URL	https://arxiv.org/abs/1909.00306v1
PDF	https://arxiv.org/pdf/1909.00306v1.pdf
PWC	https://paperswithcode.com/paper/categorical-co-frequency-analysis-clustering
Repo	https://github.com/halleewong/cofa
Framework	none

The Hanabi Challenge: A New Frontier for AI Research


Title	The Hanabi Challenge: A New Frontier for AI Research
Authors	Nolan Bard, Jakob N. Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H. Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, Iain Dunning, Shibl Mourad, Hugo Larochelle, Marc G. Bellemare, Michael Bowling
Abstract	From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay with two to five players and imperfect information. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques for such theory of mind reasoning will not only be crucial for success in Hanabi, but also in broader collaborative efforts, especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques.
Tasks	Decision Making, Game of Hanabi
Published	2019-02-01
URL	https://arxiv.org/abs/1902.00506v2
PDF	https://arxiv.org/pdf/1902.00506v2.pdf
PWC	https://paperswithcode.com/paper/the-hanabi-challenge-a-new-frontier-for-ai
Repo	https://github.com/WuTheFWasThat/hanabi.rs
Framework	none