July 29, 2019

2732 words 13 mins read

Paper Group AWR 158

Large Kernel Matters – Improve Semantic Segmentation by Global Convolutional Network. AMIDST: a Java Toolbox for Scalable Probabilistic Machine Learning. Rethinking Atrous Convolution for Semantic Image Segmentation. Feedback-prop: Convolutional Neural Network Inference under Partial Evidence. Parsing with Traces: An $O(n^4)$ Algorithm and a Struc …

Large Kernel Matters – Improve Semantic Segmentation by Global Convolutional Network


Title	Large Kernel Matters – Improve Semantic Segmentation by Global Convolutional Network
Authors	Chao Peng, Xiangyu Zhang, Gang Yu, Guiming Luo, Jian Sun
Abstract	One of recent trends [30, 31, 14] in network architec- ture design is stacking small filters (e.g., 1x1 or 3x3) in the entire network because the stacked small filters is more ef- ficient than a large kernel, given the same computational complexity. However, in the field of semantic segmenta- tion, where we need to perform dense per-pixel prediction, we find that the large kernel (and effective receptive field) plays an important role when we have to perform the clas- sification and localization tasks simultaneously. Following our design principle, we propose a Global Convolutional Network to address both the classification and localization issues for the semantic segmentation. We also suggest a residual-based boundary refinement to further refine the ob- ject boundaries. Our approach achieves state-of-art perfor- mance on two public benchmarks and significantly outper- forms previous results, 82.2% (vs 80.2%) on PASCAL VOC 2012 dataset and 76.9% (vs 71.8%) on Cityscapes dataset.
Tasks	Semantic Segmentation
Published	2017-03-08
URL	http://arxiv.org/abs/1703.02719v1
PDF	http://arxiv.org/pdf/1703.02719v1.pdf
PWC	https://paperswithcode.com/paper/large-kernel-matters-improve-semantic
Repo	https://github.com/preritj/segmentation
Framework	tf

AMIDST: a Java Toolbox for Scalable Probabilistic Machine Learning


Title	AMIDST: a Java Toolbox for Scalable Probabilistic Machine Learning
Authors	Andrés R. Masegosa, Ana M. Martínez, Darío Ramos-López, Rafael Cabañas, Antonio Salmerón, Thomas D. Nielsen, Helge Langseth, Anders L. Madsen
Abstract	The AMIDST Toolbox is a software for scalable probabilistic machine learning with a spe- cial focus on (massive) streaming data. The toolbox supports a flexible modeling language based on probabilistic graphical models with latent variables and temporal dependencies. The specified models can be learnt from large data sets using parallel or distributed implementa- tions of Bayesian learning algorithms for either streaming or batch data. These algorithms are based on a flexible variational message passing scheme, which supports discrete and continu- ous variables from a wide range of probability distributions. AMIDST also leverages existing functionality and algorithms by interfacing to software tools such as Flink, Spark, MOA, Weka, R and HUGIN. AMIDST is an open source toolbox written in Java and available at http://www.amidsttoolbox.com under the Apache Software License version 2.0.
Tasks
Published	2017-04-04
URL	http://arxiv.org/abs/1704.01427v1
PDF	http://arxiv.org/pdf/1704.01427v1.pdf
PWC	https://paperswithcode.com/paper/amidst-a-java-toolbox-for-scalable
Repo	https://github.com/amidst/jss
Framework	none

Rethinking Atrous Convolution for Semantic Image Segmentation


Title	Rethinking Atrous Convolution for Semantic Image Segmentation
Authors	Liang-Chieh Chen, George Papandreou, Florian Schroff, Hartwig Adam
Abstract	In this work, we revisit atrous convolution, a powerful tool to explicitly adjust filter’s field-of-view as well as control the resolution of feature responses computed by Deep Convolutional Neural Networks, in the application of semantic image segmentation. To handle the problem of segmenting objects at multiple scales, we design modules which employ atrous convolution in cascade or in parallel to capture multi-scale context by adopting multiple atrous rates. Furthermore, we propose to augment our previously proposed Atrous Spatial Pyramid Pooling module, which probes convolutional features at multiple scales, with image-level features encoding global context and further boost performance. We also elaborate on implementation details and share our experience on training our system. The proposed `DeepLabv3’ system significantly improves over our previous DeepLab versions without DenseCRF post-processing and attains comparable performance with other state-of-art models on the PASCAL VOC 2012 semantic image segmentation benchmark. \|
Tasks	Semantic Segmentation
Published	2017-06-17
URL	http://arxiv.org/abs/1706.05587v3
PDF	http://arxiv.org/pdf/1706.05587v3.pdf
PWC	https://paperswithcode.com/paper/rethinking-atrous-convolution-for-semantic
Repo	https://github.com/bonlime/keras-deeplab-v3-plus
Framework	tf

Feedback-prop: Convolutional Neural Network Inference under Partial Evidence


Title	Feedback-prop: Convolutional Neural Network Inference under Partial Evidence
Authors	Tianlu Wang, Kota Yamaguchi, Vicente Ordonez
Abstract	We propose an inference procedure for deep convolutional neural networks (CNNs) when partial evidence is available. Our method consists of a general feedback-based propagation approach (feedback-prop) that boosts the prediction accuracy for an arbitrary set of unknown target labels when the values for a non-overlapping arbitrary set of target labels are known. We show that existing models trained in a multi-label or multi-task setting can readily take advantage of feedback-prop without any retraining or fine-tuning. Our feedback-prop inference procedure is general, simple, reliable, and works on different challenging visual recognition tasks. We present two variants of feedback-prop based on layer-wise and residual iterative updates. We experiment using several multi-task models and show that feedback-prop is effective in all of them. Our results unveil a previously unreported but interesting dynamic property of deep CNNs. We also present an associated technical approach that takes advantage of this property for inference under partial evidence in general visual recognition tasks.
Tasks
Published	2017-10-23
URL	http://arxiv.org/abs/1710.08049v2
PDF	http://arxiv.org/pdf/1710.08049v2.pdf
PWC	https://paperswithcode.com/paper/feedback-prop-convolutional-neural-network
Repo	https://github.com/uvavision/feedbackprop
Framework	pytorch

Parsing with Traces: An $O(n^4)$ Algorithm and a Structural Representation


Title	Parsing with Traces: An $O(n^4)$ Algorithm and a Structural Representation
Authors	Jonathan K. Kummerfeld, Dan Klein
Abstract	General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons. We propose a new representation and algorithm for a class of graph structures that is flexible enough to cover almost all treebank structures, while still admitting efficient learning and inference. In particular, we consider directed, acyclic, one-endpoint-crossing graph structures, which cover most long-distance dislocation, shared argumentation, and similar tree-violating linguistic phenomena. We describe how to convert phrase structure parses, including traces, to our new representation in a reversible manner. Our dynamic program uniquely decomposes structures, is sound and complete, and covers 97.3% of the Penn English Treebank. We also implement a proof-of-concept parser that recovers a range of null elements and trace types.
Tasks	Constituency Parsing
Published	2017-07-13
URL	http://arxiv.org/abs/1707.04221v1
PDF	http://arxiv.org/pdf/1707.04221v1.pdf
PWC	https://paperswithcode.com/paper/parsing-with-traces-an-on4-algorithm-and-a-1
Repo	https://github.com/jkkummerfeld/1ec-graph-parser
Framework	none

A Transition-Based Directed Acyclic Graph Parser for UCCA


Title	A Transition-Based Directed Acyclic Graph Parser for UCCA
Authors	Daniel Hershcovich, Omri Abend, Ari Rappoport
Abstract	We present the first parser for UCCA, a cross-linguistically applicable framework for semantic representation, which builds on extensive typological work and supports rapid annotation. UCCA poses a challenge for existing parsing techniques, as it exhibits reentrancy (resulting in DAG structures), discontinuous structures and non-terminal nodes corresponding to complex semantic units. To our knowledge, the conjunction of these formal properties is not supported by any existing parser. Our transition-based parser, which uses a novel transition set and features based on bidirectional LSTMs, has value not just for UCCA parsing: its ability to handle more general graph structures can inform the development of parsers for other semantic DAG structures, and in languages that frequently use discontinuous structures.
Tasks
Published	2017-04-03
URL	http://arxiv.org/abs/1704.00552v2
PDF	http://arxiv.org/pdf/1704.00552v2.pdf
PWC	https://paperswithcode.com/paper/a-transition-based-directed-acyclic-graph
Repo	https://github.com/danielhers/tupa
Framework	none

Simple Open Stance Classification for Rumour Analysis


Title	Simple Open Stance Classification for Rumour Analysis
Authors	Ahmet Aker, Leon Derczynski, Kalina Bontcheva
Abstract	Stance classification determines the attitude, or stance, in a (typically short) text. The task has powerful applications, such as the detection of fake news or the automatic extraction of attitudes toward entities or events in the media. This paper describes a surprisingly simple and efficient classification approach to open stance classification in Twitter, for rumour and veracity classification. The approach profits from a novel set of automatically identifiable problem-specific features, which significantly boost classifier accuracy and achieve above state-of-the-art results on recent benchmark datasets. This calls into question the value of using complex sophisticated models for stance classification without first doing informed feature extraction.
Tasks
Published	2017-08-17
URL	http://arxiv.org/abs/1708.05286v2
PDF	http://arxiv.org/pdf/1708.05286v2.pdf
PWC	https://paperswithcode.com/paper/simple-open-stance-classification-for-rumour
Repo	https://github.com/radpet/fake-news-detector
Framework	tf

The Top 10 Topics in Machine Learning Revisited: A Quantitative Meta-Study


Title	The Top 10 Topics in Machine Learning Revisited: A Quantitative Meta-Study
Authors	Patrick Glauner, Manxing Du, Victor Paraschiv, Andrey Boytsov, Isabel Lopez Andrade, Jorge Meira, Petko Valtchev, Radu State
Abstract	Which topics of machine learning are most commonly addressed in research? This question was initially answered in 2007 by doing a qualitative survey among distinguished researchers. In our study, we revisit this question from a quantitative perspective. Concretely, we collect 54K abstracts of papers published between 2007 and 2016 in leading machine learning journals and conferences. We then use machine learning in order to determine the top 10 topics in machine learning. We not only include models, but provide a holistic view across optimization, data, features, etc. This quantitative approach allows reducing the bias of surveys. It reveals new and up-to-date insights into what the 10 most prolific topics in machine learning research are. This allows researchers to identify popular topics as well as new and rising topics for their research.
Tasks
Published	2017-03-29
URL	http://arxiv.org/abs/1703.10121v1
PDF	http://arxiv.org/pdf/1703.10121v1.pdf
PWC	https://paperswithcode.com/paper/the-top-10-topics-in-machine-learning
Repo	https://github.com/pglauner/MLtop10
Framework	none

Detecting Adversarial Samples from Artifacts


Title	Detecting Adversarial Samples from Artifacts
Authors	Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, Andrew B. Gardner
Abstract	Deep neural networks (DNNs) are powerful nonlinear architectures that are known to be robust to random perturbations of the input. However, these models are vulnerable to adversarial perturbations–small input changes crafted explicitly to fool the model. In this paper, we ask whether a DNN can distinguish adversarial samples from their normal and noisy counterparts. We investigate model confidence on adversarial samples by looking at Bayesian uncertainty estimates, available in dropout neural networks, and by performing density estimation in the subspace of deep features learned by the model. The result is a method for implicit adversarial detection that is oblivious to the attack algorithm. We evaluate this method on a variety of standard datasets including MNIST and CIFAR-10 and show that it generalizes well across different architectures and attacks. Our findings report that 85-93% ROC-AUC can be achieved on a number of standard classification tasks with a negative class that consists of both normal and noisy samples.
Tasks	Density Estimation
Published	2017-03-01
URL	http://arxiv.org/abs/1703.00410v3
PDF	http://arxiv.org/pdf/1703.00410v3.pdf
PWC	https://paperswithcode.com/paper/detecting-adversarial-samples-from-artifacts
Repo	https://github.com/rfeinman/detecting-adversarial-samples
Framework	tf

SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications


Title	SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications
Authors	Isabelle Augenstein, Mrinal Das, Sebastian Riedel, Lakshmi Vikraman, Andrew McCallum
Abstract	We describe the SemEval task of extracting keyphrases and relations between them from scientific documents, which is crucial for understanding which publications describe which processes, tasks and materials. Although this was a new task, we had a total of 26 submissions across 3 evaluation scenarios. We expect the task and the findings reported in this paper to be relevant for researchers working on understanding scientific content, as well as the broader knowledge base population and information extraction communities.
Tasks	Knowledge Base Population
Published	2017-04-10
URL	http://arxiv.org/abs/1704.02853v3
PDF	http://arxiv.org/pdf/1704.02853v3.pdf
PWC	https://paperswithcode.com/paper/semeval-2017-task-10-scienceie-extracting
Repo	https://github.com/LIAAD/KeywordExtractor-Datasets
Framework	none

Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval


Title	Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval
Authors	Kai Hui, Andrew Yates, Klaus Berberich, Gerard de Melo
Abstract	Neural IR models, such as DRMM and PACRR, have achieved strong results by successfully capturing relevance matching signals. We argue that the context of these matching signals is also important. Intuitively, when extracting, modeling, and combining matching signals, one would like to consider the surrounding text (local context) as well as other signals from the same document that can contribute to the overall relevance score. In this work, we highlight three potential shortcomings caused by not considering context information and propose three neural ingredients to address them: a disambiguation component, cascade k-max pooling, and a shuffling combination layer. Incorporating these components into the PACRR model yields Co-PACRR, a novel context-aware neural IR model. Extensive comparisons with established models on Trec Web Track data confirm that the proposed model can achieve superior search results. In addition, an ablation analysis is conducted to gain insights into the impact of and interactions between different components. We release our code to enable future comparisons.
Tasks	Ad-Hoc Information Retrieval
Published	2017-06-30
URL	http://arxiv.org/abs/1706.10192v3
PDF	http://arxiv.org/pdf/1706.10192v3.pdf
PWC	https://paperswithcode.com/paper/co-pacrr-a-context-aware-neural-ir-model-for
Repo	https://github.com/MatanRad/Neural-IR-Project
Framework	none

A Linear-Time Kernel Goodness-of-Fit Test


Title	A Linear-Time Kernel Goodness-of-Fit Test
Authors	Wittawat Jitkrittum, Wenkai Xu, Zoltan Szabo, Kenji Fukumizu, Arthur Gretton
Abstract	We propose a novel adaptive test of goodness-of-fit, with computational cost linear in the number of samples. We learn the test features that best indicate the differences between observed samples and a reference model, by minimizing the false negative rate. These features are constructed via Stein’s method, meaning that it is not necessary to compute the normalising constant of the model. We analyse the asymptotic Bahadur efficiency of the new test, and prove that under a mean-shift alternative, our test always has greater relative efficiency than a previous linear-time kernel test, regardless of the choice of parameters for that test. In experiments, the performance of our method exceeds that of the earlier linear-time test, and matches or exceeds the power of a quadratic-time kernel test. In high dimensions and where model structure may be exploited, our goodness of fit test performs far better than a quadratic-time two-sample test based on the Maximum Mean Discrepancy, with samples drawn from the model.
Tasks
Published	2017-05-22
URL	http://arxiv.org/abs/1705.07673v2
PDF	http://arxiv.org/pdf/1705.07673v2.pdf
PWC	https://paperswithcode.com/paper/a-linear-time-kernel-goodness-of-fit-test
Repo	https://github.com/jenninglim/model-comparison-test
Framework	none

Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes


Title	Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
Authors	Nikhil Garg, Londa Schiebinger, Dan Jurafsky, James Zou
Abstract	Word embeddings use vectors to represent words such that the geometry between vectors captures semantic relationship between the words. In this paper, we develop a framework to demonstrate how the temporal dynamics of the embedding can be leveraged to quantify changes in stereotypes and attitudes toward women and ethnic minorities in the 20th and 21st centuries in the United States. We integrate word embeddings trained on 100 years of text data with the U.S. Census to show that changes in the embedding track closely with demographic and occupation shifts over time. The embedding captures global social shifts – e.g., the women’s movement in the 1960s and Asian immigration into the U.S – and also illuminates how specific adjectives and occupations became more closely associated with certain populations over time. Our framework for temporal analysis of word embedding opens up a powerful new intersection between machine learning and quantitative social science.
Tasks	Word Embeddings
Published	2017-11-22
URL	http://arxiv.org/abs/1711.08412v1
PDF	http://arxiv.org/pdf/1711.08412v1.pdf
PWC	https://paperswithcode.com/paper/word-embeddings-quantify-100-years-of-gender
Repo	https://github.com/nikhgarg/EmbeddingDynamicStereotypes
Framework	none

The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks


Title	The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks
Authors	Maxim Berman, Amal Rannen Triki, Matthew B. Blaschko
Abstract	The Jaccard index, also referred to as the intersection-over-union score, is commonly employed in the evaluation of image segmentation results given its perceptual qualities, scale invariance - which lends appropriate relevance to small objects, and appropriate counting of false negatives, in comparison to per-pixel losses. We present a method for direct optimization of the mean intersection-over-union loss in neural networks, in the context of semantic image segmentation, based on the convex Lov'asz extension of submodular losses. The loss is shown to perform better with respect to the Jaccard index measure than the traditionally used cross-entropy loss. We show quantitative and qualitative differences between optimizing the Jaccard index per image versus optimizing the Jaccard index taken over an entire dataset. We evaluate the impact of our method in a semantic segmentation pipeline and show substantially improved intersection-over-union segmentation scores on the Pascal VOC and Cityscapes datasets using state-of-the-art deep learning segmentation architectures.
Tasks	Semantic Segmentation
Published	2017-05-24
URL	http://arxiv.org/abs/1705.08790v2
PDF	http://arxiv.org/pdf/1705.08790v2.pdf
PWC	https://paperswithcode.com/paper/the-lovasz-softmax-loss-a-tractable-surrogate
Repo	https://github.com/bermanmaxim/LovaszSoftmax
Framework	tf

ConvAMR: Abstract meaning representation parsing for legal document


Title	ConvAMR: Abstract meaning representation parsing for legal document
Authors	Lai Dac Viet, Vu Trong Sinh, Nguyen Le Minh, Ken Satoh
Abstract	Convolutional neural networks (CNN) have recently achieved remarkable performance in a wide range of applications. In this research, we equip convolutional sequence-to-sequence (seq2seq) model with an efficient graph linearization technique for abstract meaning representation parsing. Our linearization method is better than the prior method at signaling the turn of graph traveling. Additionally, convolutional seq2seq model is more appropriate and considerably faster than the recurrent neural network models in this task. Our method outperforms previous methods by a large margin on both the standard dataset LDC2014T12. Our result indicates that future works still have a room for improving parsing model using graph linearization approach.
Tasks
Published	2017-11-16
URL	http://arxiv.org/abs/1711.06141v2
PDF	http://arxiv.org/pdf/1711.06141v2.pdf
PWC	https://paperswithcode.com/paper/convamr-abstract-meaning-representation
Repo	https://github.com/laiviet/ConvAMR-torch
Framework	torch