October 21, 2019

2998 words 15 mins read

Paper Group AWR 106

Towards Large Scale Training Of Autoencoders For Collaborative Filtering. Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning. Predicting drug response of tumors from integrated genomic profiles by deep neural networks. Large Scale GAN Training for High Fidelity Natural Image Synthesis. Model-Ensemble Trust-Region Policy O …

Towards Large Scale Training Of Autoencoders For Collaborative Filtering


Title	Towards Large Scale Training Of Autoencoders For Collaborative Filtering
Authors	Abdallah Moussawi
Abstract	In this paper, we apply a mini-batch based negative sampling method to efficiently train a latent factor autoencoder model on large scale and sparse data for implicit feedback collaborative filtering. We compare our work against a state-of-the-art baseline model on different experimental datasets and show that this method can lead to a good and fast approximation of the baseline model performance. The source code is available in https://github.com/amoussawi/recoder .
Tasks
Published	2018-08-30
URL	http://arxiv.org/abs/1809.00999v3
PDF	http://arxiv.org/pdf/1809.00999v3.pdf
PWC	https://paperswithcode.com/paper/towards-large-scale-training-of-autoencoders
Repo	https://github.com/amoussawi/recoder
Framework	pytorch

Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning


Title	Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning
Authors	Yin Cui, Yang Song, Chen Sun, Andrew Howard, Serge Belongie
Abstract	Transferring the knowledge learned from large scale datasets (e.g., ImageNet) via fine-tuning offers an effective solution for domain-specific fine-grained visual categorization (FGVC) tasks (e.g., recognizing bird species or car make and model). In such scenarios, data annotation often calls for specialized domain knowledge and thus is difficult to scale. In this work, we first tackle a problem in large scale FGVC. Our method won first place in iNaturalist 2017 large scale species classification challenge. Central to the success of our approach is a training scheme that uses higher image resolution and deals with the long-tailed distribution of training data. Next, we study transfer learning via fine-tuning from large scale datasets to small scale, domain-specific FGVC datasets. We propose a measure to estimate domain similarity via Earth Mover’s Distance and demonstrate that transfer learning benefits from pre-training on a source domain that is similar to the target domain by this measure. Our proposed transfer learning outperforms ImageNet pre-training and obtains state-of-the-art results on multiple commonly used FGVC datasets.
Tasks	Fine-Grained Image Classification, Fine-Grained Visual Categorization, Transfer Learning
Published	2018-06-16
URL	http://arxiv.org/abs/1806.06193v1
PDF	http://arxiv.org/pdf/1806.06193v1.pdf
PWC	https://paperswithcode.com/paper/large-scale-fine-grained-categorization-and
Repo	https://github.com/richardaecn/cvpr18-inaturalist-transfer
Framework	tf

Predicting drug response of tumors from integrated genomic profiles by deep neural networks


Title	Predicting drug response of tumors from integrated genomic profiles by deep neural networks
Authors	Yu-Chiao Chiu, Hung-I Harry Chen, Tinghe Zhang, Songyao Zhang, Aparna Gorthi, Li-Ju Wang, Yufei Huang, Yidong Chen
Abstract	The study of high-throughput genomic profiles from a pharmacogenomics viewpoint has provided unprecedented insights into the oncogenic features modulating drug response. A recent screening of ~1,000 cancer cell lines to a collection of anti-cancer drugs illuminated the link between genotypes and vulnerability. However, due to essential differences between cell lines and tumors, the translation into predicting drug response in tumors remains challenging. Here we proposed a DNN model to predict drug response based on mutation and expression profiles of a cancer cell or a tumor. The model contains a mutation and an expression encoders pre-trained using a large pan-cancer dataset to abstract core representations of high-dimension data, followed by a drug response predictor network. Given a pair of mutation and expression profiles, the model predicts IC50 values of 265 drugs. We trained and tested the model on a dataset of 622 cancer cell lines and achieved an overall prediction performance of mean squared error at 1.96 (log-scale IC50 values). The performance was superior in prediction error or stability than two classical methods and four analog DNNs of our model. We then applied the model to predict drug response of 9,059 tumors of 33 cancer types. The model predicted both known, including EGFR inhibitors in non-small cell lung cancer and tamoxifen in ER+ breast cancer, and novel drug targets. The comprehensive analysis further revealed the molecular mechanisms underlying the resistance to a chemotherapeutic drug docetaxel in a pan-cancer setting and the anti-cancer potential of a novel agent, CX-5461, in treating gliomas and hematopoietic malignancies. Overall, our model and findings improve the prediction of drug response and the identification of novel therapeutic options.
Tasks
Published	2018-05-20
URL	http://arxiv.org/abs/1805.07702v1
PDF	http://arxiv.org/pdf/1805.07702v1.pdf
PWC	https://paperswithcode.com/paper/predicting-drug-response-of-tumors-from
Repo	https://github.com/Paureel/Thesis
Framework	none

Large Scale GAN Training for High Fidelity Natural Image Synthesis


Title	Large Scale GAN Training for High Fidelity Natural Image Synthesis
Authors	Andrew Brock, Jeff Donahue, Karen Simonyan
Abstract	Despite recent progress in generative image modeling, successfully generating high-resolution, diverse samples from complex datasets such as ImageNet remains an elusive goal. To this end, we train Generative Adversarial Networks at the largest scale yet attempted, and study the instabilities specific to such scale. We find that applying orthogonal regularization to the generator renders it amenable to a simple “truncation trick,” allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator’s input. Our modifications lead to models which set the new state of the art in class-conditional image synthesis. When trained on ImageNet at 128x128 resolution, our models (BigGANs) achieve an Inception Score (IS) of 166.5 and Frechet Inception Distance (FID) of 7.4, improving over the previous best IS of 52.52 and FID of 18.6.
Tasks	Conditional Image Generation, Image Generation
Published	2018-09-28
URL	http://arxiv.org/abs/1809.11096v2
PDF	http://arxiv.org/pdf/1809.11096v2.pdf
PWC	https://paperswithcode.com/paper/large-scale-gan-training-for-high-fidelity
Repo	https://github.com/ANIME305/Anime-GAN
Framework	tf

Model-Ensemble Trust-Region Policy Optimization


Title	Model-Ensemble Trust-Region Policy Optimization
Authors	Thanard Kurutach, Ignasi Clavera, Yan Duan, Aviv Tamar, Pieter Abbeel
Abstract	Model-free reinforcement learning (RL) methods are succeeding in a growing number of tasks, aided by recent advances in deep learning. However, they tend to suffer from high sample complexity, which hinders their use in real-world domains. Alternatively, model-based reinforcement learning promises to reduce sample complexity, but tends to require careful tuning and to date have succeeded mainly in restrictive domains where simple models are sufficient for learning. In this paper, we analyze the behavior of vanilla model-based reinforcement learning methods when deep neural networks are used to learn both the model and the policy, and show that the learned policy tends to exploit regions where insufficient data is available for the model to be learned, causing instability in training. To overcome this issue, we propose to use an ensemble of models to maintain the model uncertainty and regularize the learning process. We further show that the use of likelihood ratio derivatives yields much more stable learning than backpropagation through time. Altogether, our approach Model-Ensemble Trust-Region Policy Optimization (ME-TRPO) significantly reduces the sample complexity compared to model-free deep RL methods on challenging continuous control benchmark tasks.
Tasks	Continuous Control
Published	2018-02-28
URL	http://arxiv.org/abs/1802.10592v2
PDF	http://arxiv.org/pdf/1802.10592v2.pdf
PWC	https://paperswithcode.com/paper/model-ensemble-trust-region-policy
Repo	https://github.com/WilsonWangTHU/mbbl-metrpo
Framework	none

Shared Representational Geometry Across Neural Networks


Title	Shared Representational Geometry Across Neural Networks
Authors	Qihong Lu, Po-Hsuan Chen, Jonathan W. Pillow, Peter J. Ramadge, Kenneth A. Norman, Uri Hasson
Abstract	Different neural networks trained on the same dataset often learn similar input-output mappings with very different weights. Is there some correspondence between these neural network solutions? For linear networks, it has been shown that different instances of the same network architecture encode the same representational similarity matrix, and their neural activity patterns are connected by orthogonal transformations. However, it is unclear if this holds for non-linear networks. Using a shared response model, we show that different neural networks encode the same input examples as different orthogonal transformations of an underlying shared representation. We test this claim using both standard convolutional neural networks and residual networks on CIFAR10 and CIFAR100.
Tasks
Published	2018-11-28
URL	http://arxiv.org/abs/1811.11684v2
PDF	http://arxiv.org/pdf/1811.11684v2.pdf
PWC	https://paperswithcode.com/paper/shared-representational-geometry-across
Repo	https://github.com/qihongl/nnsrm-neurips18
Framework	none

WonDerM: Skin Lesion Classification with Fine-tuned Neural Networks


Title	WonDerM: Skin Lesion Classification with Fine-tuned Neural Networks
Authors	Yeong Chan Lee, Sang-Hyuk Jung, Hong-Hee Won
Abstract	As skin cancer is one of the most frequent cancers globally, accurate, non-invasive dermoscopy-based diagnosis becomes essential and promising. A task of the Part 3 of the ISIC Skin Image Analysis Challenge at MICCAI 2018 is to predict seven disease classes with skin lesion images, including melanoma (MEL), melanocytic nevus (NV), basal cell carcinoma (BCC), actinic keratosis / Bowen’s disease (intraepithelial carcinoma) (AKIEC), benign keratosis (solar lentigo / seborrheic keratosis / lichen planus-like keratosis) (BKL), dermatofibroma (DF) and vascular lesion (VASC) as defined by the International Dermatology Society. In this work, we design the WonDerM pipeline, that resamples the preprocessed skin lesion images, builds neural network architecture fine-tuned with segmentation task data (the Part 1), and uses an ensemble method to classify the seven skin diseases. Our model achieved an accuracy of 0.899 and 0.785 in the validation set and test set, respectively.
Tasks	Skin Lesion Classification
Published	2018-08-10
URL	https://arxiv.org/abs/1808.03426v3
PDF	https://arxiv.org/pdf/1808.03426v3.pdf
PWC	https://paperswithcode.com/paper/wonderm-skin-lesion-classification-with-fine
Repo	https://github.com/YeongChanLee/WonDerM_ISIC2018_SkinLesionAnalysis
Framework	none

Drug Similarity Integration Through Attentive Multi-view Graph Auto-Encoders


Title	Drug Similarity Integration Through Attentive Multi-view Graph Auto-Encoders
Authors	Tengfei Ma, Cao Xiao, Jiayu Zhou, Fei Wang
Abstract	Drug similarity has been studied to support downstream clinical tasks such as inferring novel properties of drugs (e.g. side effects, indications, interactions) from known properties. The growing availability of new types of drug features brings the opportunity of learning a more comprehensive and accurate drug similarity that represents the full spectrum of underlying drug relations. However, it is challenging to integrate these heterogeneous, noisy, nonlinear-related information to learn accurate similarity measures especially when labels are scarce. Moreover, there is a trade-off between accuracy and interpretability. In this paper, we propose to learn accurate and interpretable similarity measures from multiple types of drug features. In particular, we model the integration using multi-view graph auto-encoders, and add attentive mechanism to determine the weights for each view with respect to corresponding tasks and features for better interpretability. Our model has flexible design for both semi-supervised and unsupervised settings. Experimental results demonstrated significant predictive accuracy improvement. Case studies also showed better model capacity (e.g. embed node features) and interpretability.
Tasks
Published	2018-04-28
URL	http://arxiv.org/abs/1804.10850v1
PDF	http://arxiv.org/pdf/1804.10850v1.pdf
PWC	https://paperswithcode.com/paper/drug-similarity-integration-through-attentive
Repo	https://github.com/LeeJunHyun/The-Databases-for-Drug-Discovery
Framework	tf

A Question-Focused Multi-Factor Attention Network for Question Answering


Title	A Question-Focused Multi-Factor Attention Network for Question Answering
Authors	Souvik Kundu, Hwee Tou Ng
Abstract	Neural network models recently proposed for question answering (QA) primarily focus on capturing the passage-question relation. However, they have minimal capability to link relevant facts distributed across multiple sentences which is crucial in achieving deeper understanding, such as performing multi-sentence reasoning, co-reference resolution, etc. They also do not explicitly focus on the question and answer type which often plays a critical role in QA. In this paper, we propose a novel end-to-end question-focused multi-factor attention network for answer extraction. Multi-factor attentive encoding using tensor-based transformation aggregates meaningful facts even when they are located in multiple sentences. To implicitly infer the answer type, we also propose a max-attentional question aggregation mechanism to encode a question vector based on the important words in a question. During prediction, we incorporate sequence-level encoding of the first wh-word and its immediately following word as an additional source of question type information. Our proposed model achieves significant improvements over the best prior state-of-the-art results on three large-scale challenging QA datasets, namely NewsQA, TriviaQA, and SearchQA.
Tasks	Open-Domain Question Answering, Question Answering, Reading Comprehension
Published	2018-01-25
URL	http://arxiv.org/abs/1801.08290v1
PDF	http://arxiv.org/pdf/1801.08290v1.pdf
PWC	https://paperswithcode.com/paper/a-question-focused-multi-factor-attention
Repo	https://github.com/nusnlp/amanda
Framework	none

Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer


Title	Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer
Authors	David Berthelot, Colin Raffel, Aurko Roy, Ian Goodfellow
Abstract	Autoencoders provide a powerful framework for learning compressed representations by encoding all of the information needed to reconstruct a data point in a latent code. In some cases, autoencoders can “interpolate”: By decoding the convex combination of the latent codes for two datapoints, the autoencoder can produce an output which semantically mixes characteristics from the datapoints. In this paper, we propose a regularization procedure which encourages interpolated outputs to appear more realistic by fooling a critic network which has been trained to recover the mixing coefficient from interpolated data. We then develop a simple benchmark task where we can quantitatively measure the extent to which various autoencoders can interpolate and show that our regularizer dramatically improves interpolation in this setting. We also demonstrate empirically that our regularizer produces latent codes which are more effective on downstream tasks, suggesting a possible link between interpolation abilities and learning useful representations.
Tasks
Published	2018-07-19
URL	http://arxiv.org/abs/1807.07543v2
PDF	http://arxiv.org/pdf/1807.07543v2.pdf
PWC	https://paperswithcode.com/paper/understanding-and-improving-interpolation-in
Repo	https://github.com/greentfrapp/acai-berry
Framework	tf

WiRe57 : A Fine-Grained Benchmark for Open Information Extraction


Title	WiRe57 : A Fine-Grained Benchmark for Open Information Extraction
Authors	William Léchelle, Fabrizio Gotti, Philippe Langlais
Abstract	We build a reference for the task of Open Information Extraction, on five documents. We tentatively resolve a number of issues that arise, including inference and granularity. We seek to better pinpoint the requirements for the task. We produce our annotation guidelines specifying what is correct to extract and what is not. In turn, we use this reference to score existing Open IE systems. We address the non-trivial problem of evaluating the extractions produced by systems against the reference tuples, and share our evaluation script. Among seven compared extractors, we find the MinIE system to perform best.
Tasks	Open Information Extraction
Published	2018-09-24
URL	https://arxiv.org/abs/1809.08962v2
PDF	https://arxiv.org/pdf/1809.08962v2.pdf
PWC	https://paperswithcode.com/paper/wire57-a-fine-grained-benchmark-for-open
Repo	https://github.com/rali-udem/WiRe57
Framework	none

Scaling Neural Machine Translation


Title	Scaling Neural Machine Translation
Authors	Myle Ott, Sergey Edunov, David Grangier, Michael Auli
Abstract	Sequence to sequence learning models still require several days to reach state of the art performance on large benchmark datasets using a single machine. This paper shows that reduced precision and large batch training can speedup training by nearly 5x on a single 8-GPU machine with careful tuning and implementation. On WMT’14 English-German translation, we match the accuracy of Vaswani et al. (2017) in under 5 hours when training on 8 GPUs and we obtain a new state of the art of 29.3 BLEU after training for 85 minutes on 128 GPUs. We further improve these results to 29.8 BLEU by training on the much larger Paracrawl dataset. On the WMT’14 English-French task, we obtain a state-of-the-art BLEU of 43.2 in 8.5 hours on 128 GPUs.
Tasks	Machine Translation
Published	2018-06-01
URL	http://arxiv.org/abs/1806.00187v3
PDF	http://arxiv.org/pdf/1806.00187v3.pdf
PWC	https://paperswithcode.com/paper/scaling-neural-machine-translation
Repo	https://github.com/pytorch/fairseq
Framework	pytorch

Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem


Title	Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem
Authors	Matthias Hein, Maksym Andriushchenko, Julian Bitterwolf
Abstract	Classifiers used in the wild, in particular for safety-critical systems, should not only have good generalization properties but also should know when they don’t know, in particular make low confidence predictions far away from the training data. We show that ReLU type neural networks which yield a piecewise linear classifier function fail in this regard as they produce almost always high confidence predictions far away from the training data. For bounded domains like images we propose a new robust optimization technique similar to adversarial training which enforces low confidence predictions far away from the training data. We show that this technique is surprisingly effective in reducing the confidence of predictions far away from the training data while maintaining high confidence predictions and test error on the original classification task compared to standard training.
Tasks
Published	2018-12-13
URL	https://arxiv.org/abs/1812.05720v2
PDF	https://arxiv.org/pdf/1812.05720v2.pdf
PWC	https://paperswithcode.com/paper/why-relu-networks-yield-high-confidence
Repo	https://github.com/max-andr/relu_networks_overconfident
Framework	tf

Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding


Title	Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding
Authors	Tianshui Chen, Wenxi Wu, Yuefang Gao, Le Dong, Xiaonan Luo, Liang Lin
Abstract	Object categories inherently form a hierarchy with different levels of concept abstraction, especially for fine-grained categories. For example, birds (Aves) can be categorized according to a four-level hierarchy of order, family, genus, and species. This hierarchy encodes rich correlations among various categories across different levels, which can effectively regularize the semantic space and thus make prediction less ambiguous. However, previous studies of fine-grained image recognition primarily focus on categories of one certain level and usually overlook this correlation information. In this work, we investigate simultaneously predicting categories of different levels in the hierarchy and integrating this structured correlation information into the deep neural network by developing a novel Hierarchical Semantic Embedding (HSE) framework. Specifically, the HSE framework sequentially predicts the category score vector of each level in the hierarchy, from highest to lowest. At each level, it incorporates the predicted score vector of the higher level as prior knowledge to learn finer-grained feature representation. During training, the predicted score vector of the higher level is also employed to regularize label prediction by using it as soft targets of corresponding sub-categories. To evaluate the proposed framework, we organize the 200 bird species of the Caltech-UCSD birds dataset with the four-level category hierarchy and construct a large-scale butterfly dataset that also covers four level categories. Extensive experiments on these two and the newly-released VegFru datasets demonstrate the superiority of our HSE framework over the baseline methods and existing competitors.
Tasks	Fine-Grained Image Classification, Fine-Grained Image Recognition, Representation Learning
Published	2018-08-14
URL	http://arxiv.org/abs/1808.04505v1
PDF	http://arxiv.org/pdf/1808.04505v1.pdf
PWC	https://paperswithcode.com/paper/fine-grained-representation-learning-and
Repo	https://github.com/HCPLab-SYSU/HSE
Framework	pytorch

Cerebrovascular Network Segmentation on MRA Images with Deep Learning


Title	Cerebrovascular Network Segmentation on MRA Images with Deep Learning
Authors	Pedro Sanches, Cyril Meyer, Vincent Vigon, Benoît Naegel
Abstract	Deep learning has been shown to produce state of the art results in many tasks in biomedical imaging, especially in segmentation. Moreover, segmentation of the cerebrovascular structure from magnetic resonance angiography is a challenging problem because its complex geometry and topology have a large inter-patient variability. Therefore, in this work, we present a convolutional neural network approach for this problem. Particularly, a new network topology inspired by the U-net 3D and by the Inception modules, entitled Uception. In addition, a discussion about the best objective function for sparse data also guided most choices during the project. State of the art models are also implemented for a comparison purpose and final results show that the proposed architecture has the best performance in this particular context.
Tasks	Cerebrovascular Network Segmentation
Published	2018-12-04
URL	http://arxiv.org/abs/1812.01752v1
PDF	http://arxiv.org/pdf/1812.01752v1.pdf
PWC	https://paperswithcode.com/paper/cerebrovascular-network-segmentation-on-mra
Repo	https://github.com/SANCHES-Pedro/Segmentation_3D_DeepLearning
Framework	tf