October 16, 2019

2619 words 13 mins read

Paper Group ANR 1139

3D G-CNNs for Pulmonary Nodule Detection. Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization. Statistically and Computationally Efficient Variance Estimator for Kernel Ridge Regression. Halo: Learning Semantics-Aware Representations for Cross-Lingual Information Extraction. Covariance-Insured Screening. Neu …

3D G-CNNs for Pulmonary Nodule Detection


Title	3D G-CNNs for Pulmonary Nodule Detection
Authors	Marysia Winkels, Taco S. Cohen
Abstract	Convolutional Neural Networks (CNNs) require a large amount of annotated data to learn from, which is often difficult to obtain in the medical domain. In this paper we show that the sample complexity of CNNs can be significantly improved by using 3D roto-translation group convolutions (G-Convs) instead of the more conventional translational convolutions. These 3D G-CNNs were applied to the problem of false positive reduction for pulmonary nodule detection, and proved to be substantially more effective in terms of performance, sensitivity to malignant nodules, and speed of convergence compared to a strong and comparable baseline architecture with regular convolutions, data augmentation and a similar number of parameters. For every dataset size tested, the G-CNN achieved a FROC score close to the CNN trained on ten times more data.
Tasks	Data Augmentation
Published	2018-04-12
URL	http://arxiv.org/abs/1804.04656v1
PDF	http://arxiv.org/pdf/1804.04656v1.pdf
PWC	https://paperswithcode.com/paper/3d-g-cnns-for-pulmonary-nodule-detection
Repo
Framework

Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization


Title	Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization
Authors	Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, Stan Birchfield
Abstract	We present a system for training deep neural networks for object detection using synthetic images. To handle the variability in real-world data, the system relies upon the technique of domain randomization, in which the parameters of the simulator$-$such as lighting, pose, object textures, etc.$-$are randomized in non-realistic ways to force the neural network to learn the essential features of the object of interest. We explore the importance of these parameters, showing that it is possible to produce a network with compelling performance using only non-artistically-generated synthetic data. With additional fine-tuning on real data, the network yields better performance than using real data alone. This result opens up the possibility of using inexpensive synthetic data for training neural networks while avoiding the need to collect large amounts of hand-annotated real-world data or to generate high-fidelity synthetic worlds$-$both of which remain bottlenecks for many applications. The approach is evaluated on bounding box detection of cars on the KITTI dataset.
Tasks	Object Detection
Published	2018-04-18
URL	http://arxiv.org/abs/1804.06516v3
PDF	http://arxiv.org/pdf/1804.06516v3.pdf
PWC	https://paperswithcode.com/paper/training-deep-networks-with-synthetic-data
Repo
Framework

Statistically and Computationally Efficient Variance Estimator for Kernel Ridge Regression


Title	Statistically and Computationally Efficient Variance Estimator for Kernel Ridge Regression
Authors	Meimei Liu, Jean Honorio, Guang Cheng
Abstract	In this paper, we propose a random projection approach to estimate variance in kernel ridge regression. Our approach leads to a consistent estimator of the true variance, while being computationally more efficient. Our variance estimator is optimal for a large family of kernels, including cubic splines and Gaussian kernels. Simulation analysis is conducted to support our theory.
Tasks
Published	2018-09-17
URL	http://arxiv.org/abs/1809.06019v1
PDF	http://arxiv.org/pdf/1809.06019v1.pdf
PWC	https://paperswithcode.com/paper/statistically-and-computationally-efficient
Repo
Framework

Halo: Learning Semantics-Aware Representations for Cross-Lingual Information Extraction


Title	Halo: Learning Semantics-Aware Representations for Cross-Lingual Information Extraction
Authors	Hongyuan Mei, Sheng Zhang, Kevin Duh, Benjamin Van Durme
Abstract	Cross-lingual information extraction (CLIE) is an important and challenging task, especially in low resource scenarios. To tackle this challenge, we propose a training method, called Halo, which enforces the local region of each hidden state of a neural model to only generate target tokens with the same semantic structure tag. This simple but powerful technique enables a neural model to learn semantics-aware representations that are robust to noise, without introducing any extra parameter, thus yielding better generalization in both high and low resource settings.
Tasks
Published	2018-05-21
URL	http://arxiv.org/abs/1805.08271v1
PDF	http://arxiv.org/pdf/1805.08271v1.pdf
PWC	https://paperswithcode.com/paper/halo-learning-semantics-aware-representations
Repo
Framework

Covariance-Insured Screening


Title	Covariance-Insured Screening
Authors	Kevin He, Jian Kang, Hyokyoung Grace Hong, Ji Zhu, Yanming Li, Huazhen Lin, Han Xu, Yi Li
Abstract	Modern bio-technologies have produced a vast amount of high-throughput data with the number of predictors far greater than the sample size. In order to identify more novel biomarkers and understand biological mechanisms, it is vital to detect signals weakly associated with outcomes among ultrahigh-dimensional predictors. However, existing screening methods, which typically ignore correlation information, are likely to miss these weak signals. By incorporating the inter-feature dependence, we propose a covariance-insured screening methodology to identify predictors that are jointly informative but only marginally weakly associated with outcomes. The validity of the method is examined via extensive simulations and real data studies for selecting potential genetic factors related to the onset of cancer.
Tasks
Published	2018-05-17
URL	http://arxiv.org/abs/1805.06595v1
PDF	http://arxiv.org/pdf/1805.06595v1.pdf
PWC	https://paperswithcode.com/paper/covariance-insured-screening
Repo
Framework

Neural Machine Translation Decoding with Terminology Constraints


Title	Neural Machine Translation Decoding with Terminology Constraints
Authors	Eva Hasler, Adrià De Gispert, Gonzalo Iglesias, Bill Byrne
Abstract	Despite the impressive quality improvements yielded by neural machine translation (NMT) systems, controlling their translation output to adhere to user-provided terminology constraints remains an open problem. We describe our approach to constrained neural decoding based on finite-state machines and multi-stack decoding which supports target-side constraints as well as constraints with corresponding aligned input text spans. We demonstrate the performance of our framework on multiple translation tasks and motivate the need for constrained decoding with attentions as a means of reducing misplacement and duplication when translating user constraints.
Tasks	Machine Translation
Published	2018-05-09
URL	http://arxiv.org/abs/1805.03750v1
PDF	http://arxiv.org/pdf/1805.03750v1.pdf
PWC	https://paperswithcode.com/paper/neural-machine-translation-decoding-with
Repo
Framework

Towards Optimal Estimation of Bivariate Isotonic Matrices with Unknown Permutations


Title	Towards Optimal Estimation of Bivariate Isotonic Matrices with Unknown Permutations
Authors	Cheng Mao, Ashwin Pananjady, Martin J. Wainwright
Abstract	Many applications, including rank aggregation, crowd-labeling, and graphon estimation, can be modeled in terms of a bivariate isotonic matrix with unknown permutations acting on its rows and/or columns. We consider the problem of estimating an unknown matrix in this class, based on noisy observations of (possibly, a subset of) its entries. We design and analyze polynomial-time algorithms that improve upon the state of the art in two distinct metrics, showing, in particular, that minimax optimal, computationally efficient estimation is achievable in certain settings. Along the way, we prove matching upper and lower bounds on the minimax radii of certain cone testing problems, which may be of independent interest.
Tasks	Graphon Estimation
Published	2018-06-25
URL	https://arxiv.org/abs/1806.09544v2
PDF	https://arxiv.org/pdf/1806.09544v2.pdf
PWC	https://paperswithcode.com/paper/towards-optimal-estimation-of-bivariate
Repo
Framework

Data models for service failure prediction in supply-chain networks


Title	Data models for service failure prediction in supply-chain networks
Authors	Monika Sharma, Tristan Glatard, Eric Gelinas, Mariam Tagmouti, Brigitte Jaumard
Abstract	We aim to predict and explain service failures in supply-chain networks, more precisely among last-mile pickup and delivery services to customers. We analyze a dataset of 500,000 services using (1) supervised classification with Random Forests, and (2) Association Rules. Our classifier reaches an average sensitivity of 0.7 and an average specificity of 0.7 for the 5 studied types of failure. Association Rules reassert the importance of confirmation calls to prevent failures due to customers not at home, show the importance of the time window size, slack time, and geographical location of the customer for the other failure types, and highlight the effect of the retailer company on several failure types. To reduce the occurrence of service failures, our data models could be coupled to optimizers, or used to define counter-measures to be taken by human dispatchers.
Tasks
Published	2018-10-20
URL	http://arxiv.org/abs/1810.09944v1
PDF	http://arxiv.org/pdf/1810.09944v1.pdf
PWC	https://paperswithcode.com/paper/data-models-for-service-failure-prediction-in
Repo
Framework

Partially Non-Recurrent Controllers for Memory-Augmented Neural Networks


Title	Partially Non-Recurrent Controllers for Memory-Augmented Neural Networks
Authors	Naoya Taguchi, Yoshimasa Tsuruoka
Abstract	Memory-Augmented Neural Networks (MANNs) are a class of neural networks equipped with an external memory, and are reported to be effective for tasks requiring a large long-term memory and its selective use. The core module of a MANN is called a controller, which is usually implemented as a recurrent neural network (RNN) (e.g., LSTM) to enable the use of contextual information in controlling the other modules. However, such an RNN-based controller often allows a MANN to directly solve the given task by using the (small) internal memory of the controller, and prevents the MANN from making the best use of the external memory, thereby resulting in a suboptimally trained model. To address this problem, we present a novel type of RNN-based controller that is partially non-recurrent and avoids the direct use of its internal memory for solving the task, while keeping the ability of using contextual information in controlling the other modules. Our empirical experiments using Neural Turing Machines and Differentiable Neural Computers on the Toy and bAbI tasks demonstrate that the proposed controllers give substantially better results than standard RNN-based controllers.
Tasks
Published	2018-12-30
URL	http://arxiv.org/abs/1812.11485v1
PDF	http://arxiv.org/pdf/1812.11485v1.pdf
PWC	https://paperswithcode.com/paper/partially-non-recurrent-controllers-for
Repo
Framework

Designing an Effective Metric Learning Pipeline for Speaker Diarization


Title	Designing an Effective Metric Learning Pipeline for Speaker Diarization
Authors	Vivek Sivaraman Narayanaswamy, Jayaraman J. Thiagarajan, Huan Song, Andreas Spanias
Abstract	State-of-the-art speaker diarization systems utilize knowledge from external data, in the form of a pre-trained distance metric, to effectively determine relative speaker identities to unseen data. However, much of recent focus has been on choosing the appropriate feature extractor, ranging from pre-trained $i-$vectors to representations learned via different sequence modeling architectures (e.g. 1D-CNNs, LSTMs, attention models), while adopting off-the-shelf metric learning solutions. In this paper, we argue that, regardless of the feature extractor, it is crucial to carefully design a metric learning pipeline, namely the loss function, the sampling strategy and the discrimnative margin parameter, for building robust diarization systems. Furthermore, we propose to adopt a fine-grained validation process to obtain a comprehensive evaluation of the generalization power of metric learning pipelines. To this end, we measure diarization performance across different language speakers, and variations in the number of speakers in a recording. Using empirical studies, we provide interesting insights into the effectiveness of different design choices and make recommendations.
Tasks	Metric Learning, Speaker Diarization
Published	2018-11-01
URL	http://arxiv.org/abs/1811.00183v1
PDF	http://arxiv.org/pdf/1811.00183v1.pdf
PWC	https://paperswithcode.com/paper/designing-an-effective-metric-learning
Repo
Framework

A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents


Title	A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents
Authors	Heriberto Cuayáhuitl, Seonghan Ryu, Donghyeon Lee, Jihie Kim
Abstract	The amount of dialogue history to include in a conversational agent is often underestimated and/or set in an empirical and thus possibly naive way. This suggests that principled investigations into optimal context windows are urgently needed given that the amount of dialogue history and corresponding representations can play an important role in the overall performance of a conversational system. This paper studies the amount of history required by conversational agents for reliably predicting dialogue rewards. The task of dialogue reward prediction is chosen for investigating the effects of varying amounts of dialogue history and their impact on system performance. Experimental results using a dataset of 18K human-human dialogues report that lengthy dialogue histories of at least 10 sentences are preferred (25 sentences being the best in our experiments) over short ones, and that lengthy histories are useful for training dialogue reward predictors with strong positive correlations between target dialogue rewards and predicted ones.
Tasks
Published	2018-12-02
URL	http://arxiv.org/abs/1812.00350v1
PDF	http://arxiv.org/pdf/1812.00350v1.pdf
PWC	https://paperswithcode.com/paper/a-study-on-dialogue-reward-prediction-for
Repo
Framework

PepCVAE: Semi-Supervised Targeted Design of Antimicrobial Peptide Sequences


Title	PepCVAE: Semi-Supervised Targeted Design of Antimicrobial Peptide Sequences
Authors	Payel Das, Kahini Wadhawan, Oscar Chang, Tom Sercu, Cicero Dos Santos, Matthew Riemer, Vijil Chenthamarakshan, Inkit Padhi, Aleksandra Mojsilovic
Abstract	Given the emerging global threat of antimicrobial resistance, new methods for next-generation antimicrobial design are urgently needed. We report a peptide generation framework PepCVAE, based on a semi-supervised variational autoencoder (VAE) model, for designing novel antimicrobial peptide (AMP) sequences. Our model learns a rich latent space of the biological peptide context by taking advantage of abundant, unlabeled peptide sequences. The model further learns a disentangled antimicrobial attribute space by using the feedback from a jointly trained AMP classifier that uses limited labeled instances. The disentangled representation allows for controllable generation of AMPs. Extensive analysis of the PepCVAE-generated sequences reveals superior performance of our model in comparison to a plain VAE, as PepCVAE generates novel AMP sequences with higher long-range diversity, while being closer to the training distribution of biological peptides. These features are highly desired in next-generation antimicrobial design.
Tasks
Published	2018-10-17
URL	http://arxiv.org/abs/1810.07743v3
PDF	http://arxiv.org/pdf/1810.07743v3.pdf
PWC	https://paperswithcode.com/paper/pepcvae-semi-supervised-targeted-design-of
Repo
Framework

Triplet Network with Attention for Speaker Diarization


Title	Triplet Network with Attention for Speaker Diarization
Authors	Huan Song, Megan Willi, Jayaraman J. Thiagarajan, Visar Berisha, Andreas Spanias
Abstract	In automatic speech processing systems, speaker diarization is a crucial front-end component to separate segments from different speakers. Inspired by the recent success of deep neural networks (DNNs) in semantic inferencing, triplet loss-based architectures have been successfully used for this problem. However, existing work utilizes conventional i-vectors as the input representation and builds simple fully connected networks for metric learning, thus not fully leveraging the modeling power of DNN architectures. This paper investigates the importance of learning effective representations from the sequences directly in metric learning pipelines for speaker diarization. More specifically, we propose to employ attention models to learn embeddings and the metric jointly in an end-to-end fashion. Experiments are conducted on the CALLHOME conversational speech corpus. The diarization results demonstrate that, besides providing a unified model, the proposed approach achieves improved performance when compared against existing approaches.
Tasks	Metric Learning, Speaker Diarization
Published	2018-08-04
URL	http://arxiv.org/abs/1808.01535v1
PDF	http://arxiv.org/pdf/1808.01535v1.pdf
PWC	https://paperswithcode.com/paper/triplet-network-with-attention-for-speaker
Repo
Framework

Term Definitions Help Hypernymy Detection


Title	Term Definitions Help Hypernymy Detection
Authors	Wenpeng Yin, Dan Roth
Abstract	Existing methods of hypernymy detection mainly rely on statistics over a big corpus, either mining some co-occurring patterns like “animals such as cats” or embedding words of interest into context-aware vectors. These approaches are therefore limited by the availability of a large enough corpus that can cover all terms of interest and provide sufficient contextual information to represent their meaning. In this work, we propose a new paradigm, HyperDef, for hypernymy detection – expressing word meaning by encoding word definitions, along with context driven representation. This has two main benefits: (i) Definitional sentences express (sense-specific) corpus-independent meanings of words, hence definition-driven approaches enable strong generalization – once trained, the model is expected to work well in open-domain testbeds; (ii) Global context from a large corpus and definitions provide complementary information for words. Consequently, our model, HyperDef, once trained on task-agnostic data, gets state-of-the-art results in multiple benchmarks
Tasks
Published	2018-06-12
URL	http://arxiv.org/abs/1806.04532v1
PDF	http://arxiv.org/pdf/1806.04532v1.pdf
PWC	https://paperswithcode.com/paper/term-definitions-help-hypernymy-detection
Repo
Framework

Auxiliary Tasks in Multi-task Learning


Title	Auxiliary Tasks in Multi-task Learning
Authors	Lukas Liebel, Marco Körner
Abstract	Multi-task convolutional neural networks (CNNs) have shown impressive results for certain combinations of tasks, such as single-image depth estimation (SIDE) and semantic segmentation. This is achieved by pushing the network towards learning a robust representation that generalizes well to different atomic tasks. We extend this concept by adding auxiliary tasks, which are of minor relevance for the application, to the set of learned tasks. As a kind of additional regularization, they are expected to boost the performance of the ultimately desired main tasks. To study the proposed approach, we picked vision-based road scene understanding (RSU) as an exemplary application. Since multi-task learning requires specialized datasets, particularly when using extensive sets of tasks, we provide a multi-modal dataset for multi-task RSU, called synMT. More than 2.5 $\cdot$ 10^5 synthetic images, annotated with 21 different labels, were acquired from the video game Grand Theft Auto V (GTA V). Our proposed deep multi-task CNN architecture was trained on various combination of tasks using synMT. The experiments confirmed that auxiliary tasks can indeed boost network performance, both in terms of final results and training time.
Tasks	Depth Estimation, Multi-Task Learning, Scene Understanding, Semantic Segmentation
Published	2018-05-16
URL	http://arxiv.org/abs/1805.06334v2
PDF	http://arxiv.org/pdf/1805.06334v2.pdf
PWC	https://paperswithcode.com/paper/auxiliary-tasks-in-multi-task-learning
Repo
Framework