February 1, 2020

2982 words 14 mins read

Paper Group AWR 160

PIDForest: Anomaly Detection via Partial Identification. SLATE: A Super-Lightweight Annotation Tool for Experts. Improving Neural Conversational Models with Entropy-Based Data Filtering. Breaking Speech Recognizers to Imagine Lyrics. PyTorch: An Imperative Style, High-Performance Deep Learning Library. ODIN: ODE-Informed Regression for Parameter an …

PIDForest: Anomaly Detection via Partial Identification


Title	PIDForest: Anomaly Detection via Partial Identification
Authors	Parikshit Gopalan, Vatsal Sharan, Udi Wieder
Abstract	We consider the problem of detecting anomalies in a large dataset. We propose a framework called Partial Identification which captures the intuition that anomalies are easy to distinguish from the overwhelming majority of points by relatively few attribute values. Formalizing this intuition, we propose a geometric anomaly measure for a point that we call PIDScore, which measures the minimum density of data points over all subcubes containing the point. We present PIDForest: a random forest based algorithm that finds anomalies based on this definition. We show that it performs favorably in comparison to several popular anomaly detection methods, across a broad range of benchmarks. PIDForest also provides a succinct explanation for why a point is labelled anomalous, by providing a set of features and ranges for them which are relatively uncommon in the dataset.
Tasks	Anomaly Detection
Published	2019-12-08
URL	https://arxiv.org/abs/1912.03582v1
PDF	https://arxiv.org/pdf/1912.03582v1.pdf
PWC	https://paperswithcode.com/paper/pidforest-anomaly-detection-via-partial-1
Repo	https://github.com/vatsalsharan/pidforest
Framework	none

SLATE: A Super-Lightweight Annotation Tool for Experts


Title	SLATE: A Super-Lightweight Annotation Tool for Experts
Authors	Jonathan K. Kummerfeld
Abstract	Many annotation tools have been developed, covering a wide variety of tasks and providing features like user management, pre-processing, and automatic labeling. However, all of these tools use Graphical User Interfaces, and often require substantial effort to install and configure. This paper presents a new annotation tool that is designed to fill the niche of a lightweight interface for users with a terminal-based workflow. Slate supports annotation at different scales (spans of characters, tokens, and lines, or a document) and of different types (free text, labels, and links), with easily customisable keybindings, and unicode support. In a user study comparing with other tools it was consistently the easiest to install and use. Slate fills a need not met by existing systems, and has already been used to annotate two corpora, one of which involved over 250 hours of annotation effort.
Tasks
Published	2019-07-18
URL	https://arxiv.org/abs/1907.08236v1
PDF	https://arxiv.org/pdf/1907.08236v1.pdf
PWC	https://paperswithcode.com/paper/slate-a-super-lightweight-annotation-tool-for
Repo	https://github.com/jkkummerfeld/slate
Framework	none

Improving Neural Conversational Models with Entropy-Based Data Filtering


Title	Improving Neural Conversational Models with Entropy-Based Data Filtering
Authors	Richard Csaky, Patrik Purgai, Gabor Recski
Abstract	Current neural network-based conversational models lack diversity and generate boring responses to open-ended utterances. Priors such as persona, emotion, or topic provide additional information to dialog models to aid response generation, but annotating a dataset with priors is expensive and such annotations are rarely available. While previous methods for improving the quality of open-domain response generation focused on either the underlying model or the training objective, we present a method of filtering dialog datasets by removing generic utterances from training data using a simple entropy-based approach that does not require human supervision. We conduct extensive experiments with different variations of our method, and compare dialog models across 17 evaluation metrics to show that training on datasets filtered this way results in better conversational quality as chatbots learn to output more diverse responses.
Tasks
Published	2019-05-14
URL	https://arxiv.org/abs/1905.05471v3
PDF	https://arxiv.org/pdf/1905.05471v3.pdf
PWC	https://paperswithcode.com/paper/improving-neural-conversational-models-with
Repo	https://github.com/ricsinaruto/dialog-eval
Framework	none

Breaking Speech Recognizers to Imagine Lyrics


Title	Breaking Speech Recognizers to Imagine Lyrics
Authors	Jon Gillick, David Bamman
Abstract	We introduce a new method for generating text, and in particular song lyrics, based on the speech-like acoustic qualities of a given audio file. We repurpose a vocal source separation algorithm and an acoustic model trained to recognize isolated speech, instead inputting instrumental music or environmental sounds. Feeding the “mistakes” of the vocal separator into the recognizer, we obtain a transcription of words \emph{imagined} to be spoken in the input audio. We describe the key components of our approach, present initial analysis, and discuss the potential of the method for machine-in-the-loop collaboration in creative applications.
Tasks
Published	2019-12-15
URL	https://arxiv.org/abs/1912.06979v1
PDF	https://arxiv.org/pdf/1912.06979v1.pdf
PWC	https://paperswithcode.com/paper/breaking-speech-recognizers-to-imagine-lyrics
Repo	https://github.com/jrgillick/imagined-lyrics
Framework	none

PyTorch: An Imperative Style, High-Performance Deep Learning Library


Title	PyTorch: An Imperative Style, High-Performance Deep Learning Library
Authors	Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, Soumith Chintala
Abstract	Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.
Tasks
Published	2019-12-03
URL	https://arxiv.org/abs/1912.01703v1
PDF	https://arxiv.org/pdf/1912.01703v1.pdf
PWC	https://paperswithcode.com/paper/pytorch-an-imperative-style-high-performance-1
Repo	https://github.com/pytorch/pytorch
Framework	pytorch

ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems


Title	ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems
Authors	Philippe Wenk, Gabriele Abbati, Michael A Osborne, Bernhard Schölkopf, Andreas Krause, Stefan Bauer
Abstract	Parameter inference in ordinary differential equations is an important problem in many applied sciences and in engineering, especially in a data-scarce setting. In this work, we introduce a novel generative modeling approach based on constrained Gaussian processes and leverage it to build a computationally and data efficient algorithm for state and parameter inference. In an extensive set of experiments, our approach outperforms the current state of the art for parameter inference both in terms of accuracy and computational cost. It also shows promising results for the much more challenging problem of model selection.
Tasks	Gaussian Processes, Model Selection
Published	2019-02-17
URL	https://arxiv.org/abs/1902.06278v3
PDF	https://arxiv.org/pdf/1902.06278v3.pdf
PWC	https://paperswithcode.com/paper/odin-ode-informed-regression-for-parameter
Repo	https://github.com/gabb7/ODIN
Framework	tf

Lagging Inference Networks and Posterior Collapse in Variational Autoencoders


Title	Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
Authors	Junxian He, Daniel Spokoyny, Graham Neubig, Taylor Berg-Kirkpatrick
Abstract	The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique. By using a neural inference network to approximate the model’s posterior on latent variables, VAEs efficiently parameterize a lower bound on marginal data likelihood that can be optimized directly via gradient methods. In practice, however, VAE training often results in a degenerate local optimum known as “posterior collapse” where the model learns to ignore the latent variable and the approximate posterior mimics the prior. In this paper, we investigate posterior collapse from the perspective of training dynamics. We find that during the initial stages of training the inference network fails to approximate the model’s true posterior, which is a moving target. As a result, the model is encouraged to ignore the latent encoding and posterior collapse occurs. Based on this observation, we propose an extremely simple modification to VAE training to reduce inference lag: depending on the model’s current mutual information between latent variable and observation, we aggressively optimize the inference network before performing each model update. Despite introducing neither new model components nor significant complexity over basic VAE, our approach is able to avoid the problem of collapse that has plagued a large amount of previous work. Empirically, our approach outperforms strong autoregressive baselines on text and image benchmarks in terms of held-out likelihood, and is competitive with more complex techniques for avoiding collapse while being substantially faster.
Tasks	Text Generation
Published	2019-01-16
URL	http://arxiv.org/abs/1901.05534v2
PDF	http://arxiv.org/pdf/1901.05534v2.pdf
PWC	https://paperswithcode.com/paper/lagging-inference-networks-and-posterior
Repo	https://github.com/jxhe/vae-lagging-encoder
Framework	pytorch

A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software


Title	A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software
Authors	Serena E. Ponta, Henrik Plate, Antonino Sabetta, Michele Bezzi, Cédric Dangremont
Abstract	Advancing our understanding of software vulnerabilities, automating their identification, the analysis of their impact, and ultimately their mitigation is necessary to enable the development of software that is more secure. While operating a vulnerability assessment tool that we developed and that is currently used by hundreds of development units at SAP, we manually collected and curated a dataset of vulnerabilities of open-source software and the commits fixing them. The data was obtained both from the National Vulnerability Database (NVD) and from project-specific Web resources that we monitor on a continuous basis. From that data, we extracted a dataset that maps 624 publicly disclosed vulnerabilities affecting 205 distinct open-source Java projects, used in SAP products or internal tools, onto the 1282 commits that fix them. Out of 624 vulnerabilities, 29 do not have a CVE identifier at all and 46, which do have a CVE identifier assigned by a numbering authority, are not available in the NVD yet. The dataset is released under an open-source license, together with supporting scripts that allow researchers to automatically retrieve the actual content of the commits from the corresponding repositories and to augment the attributes available for each instance. Also, these scripts allow to complement the dataset with additional instances that are not security fixes (which is useful, for example, in machine learning applications). Our dataset has been successfully used to train classifiers that could automatically identify security-relevant commits in code repositories. The release of this dataset and the supporting code as open-source will allow future research to be based on data of industrial relevance; also, it represents a concrete step towards making the maintenance of this dataset a shared effort involving open-source communities, academia, and the industry.
Tasks
Published	2019-02-07
URL	http://arxiv.org/abs/1902.02595v3
PDF	http://arxiv.org/pdf/1902.02595v3.pdf
PWC	https://paperswithcode.com/paper/a-manually-curated-dataset-of-fixes-to
Repo	https://github.com/copernico/msr2019
Framework	none

Extreme Relative Pose Network under Hybrid Representations


Title	Extreme Relative Pose Network under Hybrid Representations
Authors	Zhenpei Yang, Siming Yan, Qixing Huang
Abstract	In this paper, we introduce a novel RGB-D based relative pose estimation approach that is suitable for small-overlapping or non-overlapping scans and can output multiple relative poses. Our method performs scene completion and matches the completed scans. However, instead of using a fixed representation for completion, the key idea is to utilize hybrid representations that combine 360-image, 2D image-based layout, and planar patches. This approach offers adaptively feature representations for relative pose estimation. Besides, we introduce a global-2-local matching procedure, which utilizes initial relative poses obtained during the global phase to detect and then integrate geometric relations for pose refinement. Experimental results justify the potential of this approach across a wide range of benchmark datasets. For example, on ScanNet, the rotation translation errors of the top-1/top-5 predictions of our approach are 34.9/0.69m and 19.6/0.57m, respectively. Our approach also considerably boosts the performance of multi-scan reconstruction in few-view reconstruction settings.
Tasks	Pose Estimation
Published	2019-12-25
URL	https://arxiv.org/abs/1912.11695v1
PDF	https://arxiv.org/pdf/1912.11695v1.pdf
PWC	https://paperswithcode.com/paper/extreme-relative-pose-network-under-hybrid
Repo	https://github.com/SimingYan/Hybrid_Relative_Pose
Framework	pytorch

DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation


Title	DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation
Authors	Hanchao Li, Pengfei Xiong, Haoqiang Fan, Jian Sun
Abstract	This paper introduces an extremely efficient CNN architecture named DFANet for semantic segmentation under resource constraints. Our proposed network starts from a single lightweight backbone and aggregates discriminative features through sub-network and sub-stage cascade respectively. Based on the multi-scale feature propagation, DFANet substantially reduces the number of parameters, but still obtains sufficient receptive field and enhances the model learning ability, which strikes a balance between the speed and segmentation performance. Experiments on Cityscapes and CamVid datasets demonstrate the superior performance of DFANet with 8$\times$ less FLOPs and 2$\times$ faster than the existing state-of-the-art real-time semantic segmentation methods while providing comparable accuracy. Specifically, it achieves 70.3% Mean IOU on the Cityscapes test dataset with only 1.7 GFLOPs and a speed of 160 FPS on one NVIDIA Titan X card, and 71.3% Mean IOU with 3.4 GFLOPs while inferring on a higher resolution image.
Tasks	Real-Time Semantic Segmentation, Semantic Segmentation
Published	2019-04-03
URL	http://arxiv.org/abs/1904.02216v1
PDF	http://arxiv.org/pdf/1904.02216v1.pdf
PWC	https://paperswithcode.com/paper/dfanet-deep-feature-aggregation-for-real-time
Repo	https://github.com/j-a-lin/DFANet_PyTorch
Framework	pytorch

Representation Learning for Heterogeneous Information Networks via Embedding Events


Title	Representation Learning for Heterogeneous Information Networks via Embedding Events
Authors	Guoji Fu, Bo Yuan, Qiqi Duan, Xin Yao
Abstract	Network representation learning (NRL) has been widely used to help analyze large-scale networks through mapping original networks into a low-dimensional vector space. However, existing NRL methods ignore the impact of properties of relations on the object relevance in heterogeneous information networks (HINs). To tackle this issue, this paper proposes a new NRL framework, called Event2vec, for HINs to consider both quantities and properties of relations during the representation learning process. Specifically, an event (i.e., a complete semantic unit) is used to represent the relation among multiple objects, and both event-driven first-order and second-order proximities are defined to measure the object relevance according to the quantities and properties of relations. We theoretically prove how event-driven proximities can be preserved in the embedding space by Event2vec, which utilizes event embeddings to facilitate learning the object embeddings. Experimental studies demonstrate the advantages of Event2vec over state-of-the-art algorithms on four real-world datasets and three network analysis tasks (including network reconstruction, link prediction, and node classification).
Tasks	Link Prediction, Node Classification, Representation Learning
Published	2019-01-29
URL	http://arxiv.org/abs/1901.10234v2
PDF	http://arxiv.org/pdf/1901.10234v2.pdf
PWC	https://paperswithcode.com/paper/representation-learning-for-heterogeneous
Repo	https://github.com/fuguoji/Event2vec
Framework	tf

GEN: Highly Efficient SMILES Explorer Using Autodidactic Generative Examination Networks


Title	GEN: Highly Efficient SMILES Explorer Using Autodidactic Generative Examination Networks
Authors	Ruud van Deursen, Peter Ertl, Igor V. Tetko, Guillaume Godin
Abstract	Recurrent neural networks have been widely used to generate millions of de novo molecules in a known chemical space. These deep generative models are typically setup with LSTM or GRU units and trained with canonical SMILEs. In this study, we introduce a new robust architecture, Generative Examination Networks GEN, based on bidirectional RNNs with concatenated sub-models to learn and generate molecular SMILES with a trained target space. GENs autonomously learn the target space in a few epochs while being subjected to an independent online examination mechanism to measure the quality of the generated set. Here we have used online statistical quality control (SQC) on the percentage of valid molecules SMILES as an examination measure to select the earliest available stable model weights. Very high levels of valid SMILES (95-98%) can be generated using multiple parallel encoding layers in combination with SMILES augmentation using unrestricted SMILES randomization. Our architecture combines an excellent novelty rate (85-90%) while generating SMILES with a strong conservation of the property space (95-99%). Our flexible examination mechanism is open to other quality criteria.
Tasks
Published	2019-09-10
URL	https://arxiv.org/abs/1909.04825v1
PDF	https://arxiv.org/pdf/1909.04825v1.pdf
PWC	https://paperswithcode.com/paper/gen-highly-efficient-smiles-explorer-using
Repo	https://github.com/RuudFirsa/Smiles-GEN
Framework	tf

scikit-hubness: Hubness Reduction and Approximate Neighbor Search


Title	scikit-hubness: Hubness Reduction and Approximate Neighbor Search
Authors	Roman Feldbauer, Thomas Rattei, Arthur Flexer
Abstract	This paper introduces scikit-hubness, a Python package for efficient nearest neighbor search in high-dimensional spaces. Hubness is an aspect of the curse of dimensionality, and is known to impair various learning tasks, including classification, clustering, and visualization. scikit-hubness provides algorithms for hubness analysis (“Is my data affected by hubness?"), hubness reduction (“How can we improve neighbor retrieval in high dimensions?"), and approximate neighbor search (“Does it work for large data sets?"). It is integrated into the scikit-learn environment, enabling rapid adoption by Python-based machine learning researchers and practitioners. Users will find all functionality of the scikit-learn neighbors package, plus additional support for transparent hubness reduction and approximate nearest neighbor search. scikit-hubness is developed using several quality assessment tools and principles, such as PEP8 compliance, unit tests with high code coverage, continuous integration on all major platforms (Linux, MacOS, Windows), and additional checks by LGTM. The source code is available at https://github.com/VarIr/scikit-hubness under the BSD 3-clause license. Install from the Python package index with $ pip install scikit-hubness.
Tasks
Published	2019-12-02
URL	https://arxiv.org/abs/1912.00706v1
PDF	https://arxiv.org/pdf/1912.00706v1.pdf
PWC	https://paperswithcode.com/paper/scikit-hubness-hubness-reduction-and
Repo	https://github.com/VarIr/scikit-hubness
Framework	none

Sufficient Representations for Categorical Variables


Title	Sufficient Representations for Categorical Variables
Authors	Jonathan Johannemann, Vitor Hadad, Susan Athey, Stefan Wager
Abstract	Many learning algorithms require categorical data to be transformed into real vectors before it can be used as input. Often, categorical variables are encoded as one-hot (or dummy) vectors. However, this mode of representation can be wasteful since it adds many low-signal regressors, especially when the number of unique categories is large. In this paper, we investigate simple alternative solutions for universally consistent estimators that rely on lower-dimensional real-valued representations of categorical variables that are “sufficient” in the sense that no predictive information is lost. We then compare preexisting and proposed methods on simulated and observational datasets.
Tasks
Published	2019-08-26
URL	https://arxiv.org/abs/1908.09874v2
PDF	https://arxiv.org/pdf/1908.09874v2.pdf
PWC	https://paperswithcode.com/paper/sufficient-representations-for-categorical
Repo	https://github.com/atecon/CategoryEncoders
Framework	none

Natural Adversarial Examples


Title	Natural Adversarial Examples
Authors	Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, Dawn Song
Abstract	We introduce natural adversarial examples – real-world, unmodified, and naturally occurring examples that cause classifier accuracy to significantly degrade. We curate 7,500 natural adversarial examples and release them in an ImageNet classifier test set that we call ImageNet-A. This dataset serves as a new way to measure classifier robustness. Like l_p adversarial examples, ImageNet-A examples successfully transfer to unseen or black-box classifiers. For example, on ImageNet-A a DenseNet-121 obtains around 2% accuracy, an accuracy drop of approximately 90%. Recovering this accuracy is not simple because ImageNet-A examples exploit deep flaws in current classifiers including their over-reliance on color, texture, and background cues. We observe that popular training techniques for improving robustness have little effect, but we show that some architectural changes can enhance robustness to natural adversarial examples. Future research is required to enable robust generalization to this hard ImageNet test set.
Tasks	Adversarial Attack
Published	2019-07-16
URL	https://arxiv.org/abs/1907.07174v3
PDF	https://arxiv.org/pdf/1907.07174v3.pdf
PWC	https://paperswithcode.com/paper/natural-adversarial-examples
Repo	https://github.com/hendrycks/natural-adv-examples
Framework	pytorch