Paper Group AWR 11
Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images. Radio Galaxy Morphology Generation Using DNN Autoencoder and Gaussian Mixture Models. On Tree-Based Neural Sentence Modeling. From 2D to 3D Geodesic-based Garment Matching. Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning. Addressing the Fundamenta …
Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images
| Title | Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images |
| Authors | Christoph Baur, Benedikt Wiestler, Shadi Albarqouni, Nassir Navab |
| Abstract | Reliably modeling normality and differentiating abnormal appearances from normal cases is a very appealing approach for detecting pathologies in medical images. A plethora of such unsupervised anomaly detection approaches has been made in the medical domain, based on statistical methods, content-based retrieval, clustering and recently also deep learning. Previous approaches towards deep unsupervised anomaly detection model patches of normal anatomy with variants of Autoencoders or GANs, and detect anomalies either as outliers in the learned feature space or from large reconstruction errors. In contrast to these patch-based approaches, we show that deep spatial autoencoding models can be efficiently used to capture normal anatomical variability of entire 2D brain MR images. A variety of experiments on real MR data containing MS lesions corroborates our hypothesis that we can detect and even delineate anomalies in brain MR images by simply comparing input images to their reconstruction. Results show that constraints on the latent space and adversarial training can further improve the segmentation performance over standard deep representation learning. |
| Tasks | Anomaly Detection, Representation Learning, Unsupervised Anomaly Detection |
| Published | 2018-04-12 |
| URL | http://arxiv.org/abs/1804.04488v1 |
| http://arxiv.org/pdf/1804.04488v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-autoencoding-models-for-unsupervised |
| Repo | https://github.com/bumuckl/AutoencodersForUnsupervisedAnomalyDetection |
| Framework | tf |
Radio Galaxy Morphology Generation Using DNN Autoencoder and Gaussian Mixture Models
| Title | Radio Galaxy Morphology Generation Using DNN Autoencoder and Gaussian Mixture Models |
| Authors | Zhixian Ma, Jie Zhu, Weitian Li, Haiguang Xu |
| Abstract | The morphology of a radio galaxy is highly affected by its central active galactic nuclei (AGN), which is studied to reveal the evolution of the super massive black hole (SMBH). In this work, we propose a morphology generation framework for two typical radio galaxies namely Fanaroff-Riley type-I (FRI) and type-II (FRII) with deep neural network based autoencoder (DNNAE) and Gaussian mixture models (GMMs). The encoder and decoder subnets in the DNNAE are symmetric aside a fully-connected layer namely code layer hosting the extracted feature vectors. By randomly generating the feature vectors later with a three-component Gaussian Mixture models, new FRI or FRII radio galaxy morphologies are simulated. Experiments were demonstrated on real radio galaxy images, where we discussed the length of feature vectors, selection of lost functions, and made comparisons on batch normalization and dropout techniques for training the network. The results suggest a high efficiency and performance of our morphology generation framework. Code is available at: https://github.com/myinxd/dnnae-gmm. |
| Tasks | |
| Published | 2018-06-01 |
| URL | http://arxiv.org/abs/1806.00398v1 |
| http://arxiv.org/pdf/1806.00398v1.pdf | |
| PWC | https://paperswithcode.com/paper/radio-galaxy-morphology-generation-using-dnn |
| Repo | https://github.com/myinxd/dnnae-gmm |
| Framework | tf |
On Tree-Based Neural Sentence Modeling
| Title | On Tree-Based Neural Sentence Modeling |
| Authors | Haoyue Shi, Hao Zhou, Jiaze Chen, Lei Li |
| Abstract | Neural networks with tree-based sentence encoders have shown better results on many downstream tasks. Most of existing tree-based encoders adopt syntactic parsing trees as the explicit structure prior. To study the effectiveness of different tree structures, we replace the parsing trees with trivial trees (i.e., binary balanced tree, left-branching tree and right-branching tree) in the encoders. Though trivial trees contain no syntactic information, those encoders get competitive or even better results on all of the ten downstream tasks we investigated. This surprising result indicates that explicit syntax guidance may not be the main contributor to the superior performances of tree-based neural sentence modeling. Further analysis show that tree modeling gives better results when crucial words are closer to the final representation. Additional experiments give more clues on how to design an effective tree-based encoder. Our code is open-source and available at https://github.com/ExplorerFreda/TreeEnc. |
| Tasks | Sentiment Analysis, Text Classification |
| Published | 2018-08-29 |
| URL | http://arxiv.org/abs/1808.09644v1 |
| http://arxiv.org/pdf/1808.09644v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-tree-based-neural-sentence-modeling |
| Repo | https://github.com/ExplorerFreda/TreeEnc |
| Framework | pytorch |
From 2D to 3D Geodesic-based Garment Matching
| Title | From 2D to 3D Geodesic-based Garment Matching |
| Authors | Meysam Madadi, Egils Avots, Sergio Escalera, Jordi Gonzalez, Xavier Baro, Gholamreza Anbarjafari |
| Abstract | A new approach for 2D to 3D garment retexturing is proposed based on Gaussian mixture models and thin plate splines (TPS). An automatically segmented garment of an individual is matched to a new source garment and rendered, resulting in augmented images in which the target garment has been retextured by using the texture of the source garment. We divide the problem into garment boundary matching based on Gaussian mixture models and then interpolate inner points using surface topology extracted through geodesic paths, which leads to a more realistic result than standard approaches. We evaluated and compared our system quantitatively by mean square error (MSE) and qualitatively using the mean opinion score (MOS), showing the benefits of the proposed methodology on our gathered dataset. |
| Tasks | |
| Published | 2018-09-21 |
| URL | http://arxiv.org/abs/1809.08064v1 |
| http://arxiv.org/pdf/1809.08064v1.pdf | |
| PWC | https://paperswithcode.com/paper/from-2d-to-3d-geodesic-based-garment-matching |
| Repo | https://github.com/bing-jian/gmmreg |
| Framework | none |
Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
| Title | Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning |
| Authors | Supasorn Suwajanakorn, Noah Snavely, Jonathan Tompson, Mohammad Norouzi |
| Abstract | This paper presents KeypointNet, an end-to-end geometric reasoning framework to learn an optimal set of category-specific 3D keypoints, along with their detectors. Given a single image, KeypointNet extracts 3D keypoints that are optimized for a downstream task. We demonstrate this framework on 3D pose estimation by proposing a differentiable objective that seeks the optimal set of keypoints for recovering the relative pose between two views of an object. Our model discovers geometrically and semantically consistent keypoints across viewing angles and instances of an object category. Importantly, we find that our end-to-end framework using no ground-truth keypoint annotations outperforms a fully supervised baseline using the same neural network architecture on the task of pose estimation. The discovered 3D keypoints on the car, chair, and plane categories of ShapeNet are visualized at http://keypointnet.github.io/. |
| Tasks | 3D Pose Estimation, Pose Estimation |
| Published | 2018-07-05 |
| URL | http://arxiv.org/abs/1807.03146v2 |
| http://arxiv.org/pdf/1807.03146v2.pdf | |
| PWC | https://paperswithcode.com/paper/discovery-of-latent-3d-keypoints-via-end-to |
| Repo | https://github.com/tensorflow/models/tree/master/research/keypointnet |
| Framework | tf |
Addressing the Fundamental Tension of PCGML with Discriminative Learning
| Title | Addressing the Fundamental Tension of PCGML with Discriminative Learning |
| Authors | Isaac Karth, Adam M. Smith |
| Abstract | Procedural content generation via machine learning (PCGML) is typically framed as the task of fitting a generative model to full-scale examples of a desired content distribution. This approach presents a fundamental tension: the more design effort expended to produce detailed training examples for shaping a generator, the lower the return on investment from applying PCGML in the first place. In response, we propose the use of discriminative models (which capture the validity of a design rather the distribution of the content) trained on positive and negative examples. Through a modest modification of WaveFunctionCollapse, a commercially-adopted PCG approach that we characterize as using elementary machine learning, we demonstrate a new mode of control for learning-based generators. We demonstrate how an artist might craft a focused set of additional positive and negative examples by critique of the generator’s previous outputs. This interaction mode bridges PCGML with mixed-initiative design assistance tools by working with a machine to define a space of valid designs rather than just one new design. |
| Tasks | |
| Published | 2018-09-10 |
| URL | http://arxiv.org/abs/1809.04432v1 |
| http://arxiv.org/pdf/1809.04432v1.pdf | |
| PWC | https://paperswithcode.com/paper/addressing-the-fundamental-tension-of-pcgml |
| Repo | https://github.com/mxgmn/WaveFunctionCollapse |
| Framework | none |
Paraphrase to Explicate: Revealing Implicit Noun-Compound Relations
| Title | Paraphrase to Explicate: Revealing Implicit Noun-Compound Relations |
| Authors | Vered Shwartz, Ido Dagan |
| Abstract | Revealing the implicit semantic relation between the constituents of a noun-compound is important for many NLP applications. It has been addressed in the literature either as a classification task to a set of pre-defined relations or by producing free text paraphrases explicating the relations. Most existing paraphrasing methods lack the ability to generalize, and have a hard time interpreting infrequent or new noun-compounds. We propose a neural model that generalizes better by representing paraphrases in a continuous space, generalizing for both unseen noun-compounds and rare paraphrases. Our model helps improving performance on both the noun-compound paraphrasing and classification tasks. |
| Tasks | |
| Published | 2018-05-07 |
| URL | http://arxiv.org/abs/1805.02442v1 |
| http://arxiv.org/pdf/1805.02442v1.pdf | |
| PWC | https://paperswithcode.com/paper/paraphrase-to-explicate-revealing-implicit |
| Repo | https://github.com/vered1986/panic |
| Framework | none |
DenseFuse: A Fusion Approach to Infrared and Visible Images
| Title | DenseFuse: A Fusion Approach to Infrared and Visible Images |
| Authors | Hui Li, Xiao-Jun Wu |
| Abstract | In this paper, we present a novel deep learning architecture for infrared and visible images fusion problem. In contrast to conventional convolutional networks, our encoding network is combined by convolutional layers, fusion layer and dense block in which the output of each layer is connected to every other layer. We attempt to use this architecture to get more useful features from source images in encoding process. And two fusion layers(fusion strategies) are designed to fuse these features. Finally, the fused image is reconstructed by decoder. Compared with existing fusion methods, the proposed fusion method achieves state-of-the-art performance in objective and subjective assessment. Code and pre-trained models are available at https://github.com/hli1221/imagefusion_densefuse |
| Tasks | |
| Published | 2018-04-23 |
| URL | http://arxiv.org/abs/1804.08361v9 |
| http://arxiv.org/pdf/1804.08361v9.pdf | |
| PWC | https://paperswithcode.com/paper/densefuse-a-fusion-approach-to-infrared-and |
| Repo | https://github.com/exceptionLi/imagefusion_densefuse |
| Framework | tf |
Learning an MR acquisition-invariant representation using Siamese neural networks
| Title | Learning an MR acquisition-invariant representation using Siamese neural networks |
| Authors | Wouter M. Kouw, Marco Loog, Wilbert Bartels, Adriënne M. Mendrik |
| Abstract | Generalization of voxelwise classifiers is hampered by differences between MRI-scanners, e.g. different acquisition protocols and field strengths. To address this limitation, we propose a Siamese neural network (MRAI-NET) that extracts acquisition-invariant feature vectors. These can consequently be used by task-specific methods, such as voxelwise classifiers for tissue segmentation. MRAI-NET is tested on both simulated and real patient data. Experiments show that MRAI-NET outperforms voxelwise classifiers trained on the source or target scanner data when a small number of labeled samples is available. |
| Tasks | |
| Published | 2018-10-17 |
| URL | http://arxiv.org/abs/1810.07430v1 |
| http://arxiv.org/pdf/1810.07430v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-an-mr-acquisition-invariant |
| Repo | https://github.com/wmkouw/mrai-net |
| Framework | none |
Understanding Individual Decisions of CNNs via Contrastive Backpropagation
| Title | Understanding Individual Decisions of CNNs via Contrastive Backpropagation |
| Authors | Jindong Gu, Yinchong Yang, Volker Tresp |
| Abstract | A number of backpropagation-based approaches such as DeConvNets, vanilla Gradient Visualization and Guided Backpropagation have been proposed to better understand individual decisions of deep convolutional neural networks. The saliency maps produced by them are proven to be non-discriminative. Recently, the Layer-wise Relevance Propagation (LRP) approach was proposed to explain the classification decisions of rectifier neural networks. In this work, we evaluate the discriminativeness of the generated explanations and analyze the theoretical foundation of LRP, i.e. Deep Taylor Decomposition. The experiments and analysis conclude that the explanations generated by LRP are not class-discriminative. Based on LRP, we propose Contrastive Layer-wise Relevance Propagation (CLRP), which is capable of producing instance-specific, class-discriminative, pixel-wise explanations. In the experiments, we use the CLRP to explain the decisions and understand the difference between neurons in individual classification decisions. We also evaluate the explanations quantitatively with a Pointing Game and an ablation study. Both qualitative and quantitative evaluations show that the CLRP generates better explanations than the LRP. The code is available. |
| Tasks | |
| Published | 2018-12-05 |
| URL | https://arxiv.org/abs/1812.02100v2 |
| https://arxiv.org/pdf/1812.02100v2.pdf | |
| PWC | https://paperswithcode.com/paper/understanding-individual-decisions-of-cnns |
| Repo | https://github.com/Jindong-Explainable-AI/Contrastive-LRP |
| Framework | pytorch |
Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization
| Title | Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization |
| Authors | James Vuckovic |
| Abstract | We introduce Kalman Gradient Descent, a stochastic optimization algorithm that uses Kalman filtering to adaptively reduce gradient variance in stochastic gradient descent by filtering the gradient estimates. We present both a theoretical analysis of convergence in a non-convex setting and experimental results which demonstrate improved performance on a variety of machine learning areas including neural networks and black box variational inference. We also present a distributed version of our algorithm that enables large-dimensional optimization, and we extend our algorithm to SGD with momentum and RMSProp. |
| Tasks | Stochastic Optimization |
| Published | 2018-10-29 |
| URL | http://arxiv.org/abs/1810.12273v1 |
| http://arxiv.org/pdf/1810.12273v1.pdf | |
| PWC | https://paperswithcode.com/paper/kalman-gradient-descent-adaptive-variance |
| Repo | https://github.com/jamesvuc/KGD |
| Framework | tf |
Graph Pattern Mining and Learning through User-defined Relations (Extended Version)
| Title | Graph Pattern Mining and Learning through User-defined Relations (Extended Version) |
| Authors | Carlos H. C. Teixeira, Leonardo Cotta, Bruno Ribeiro, Wagner Meira Jr |
| Abstract | In this work we propose R-GPM, a parallel computing framework for graph pattern mining (GPM) through a user-defined subgraph relation. More specifically, we enable the computation of statistics of patterns through their subgraph classes, generalizing traditional GPM methods. R-GPM provides efficient estimators for these statistics by employing a MCMC sampling algorithm combined with several optimizations. We provide both theoretical guarantees and empirical evaluations of our estimators in application scenarios such as stochastic optimization of deep high-order graph neural network models and pattern (motif) counting. We also propose and evaluate optimizations that enable improvements of our estimators accuracy, while reducing their computational costs in up to 3-orders-of-magnitude. Finally,we show that R-GPM is scalable, providing near-linear speedups on 44 cores in all of our tests. |
| Tasks | Stochastic Optimization |
| Published | 2018-09-14 |
| URL | http://arxiv.org/abs/1809.05241v1 |
| http://arxiv.org/pdf/1809.05241v1.pdf | |
| PWC | https://paperswithcode.com/paper/graph-pattern-mining-and-learning-through |
| Repo | https://github.com/dccspeed/rgpm |
| Framework | none |
Stochastic Gradient Descent with Biased but Consistent Gradient Estimators
| Title | Stochastic Gradient Descent with Biased but Consistent Gradient Estimators |
| Authors | Jie Chen, Ronny Luss |
| Abstract | Stochastic gradient descent (SGD), which dates back to the 1950s, is one of the most popular and effective approaches for performing stochastic optimization. Research on SGD resurged recently in machine learning for optimizing convex loss functions and training nonconvex deep neural networks. The theory assumes that one can easily compute an unbiased gradient estimator, which is usually the case due to the sample average nature of empirical risk minimization. There exist, however, many scenarios (e.g., graphs) where an unbiased estimator may be as expensive to compute as the full gradient because training examples are interconnected. Recently, Chen et al. (2018) proposed using a consistent gradient estimator as an economic alternative. Encouraged by empirical success, we show, in a general setting, that consistent estimators result in the same convergence behavior as do unbiased ones. Our analysis covers strongly convex, convex, and nonconvex objectives. We verify the results with illustrative experiments on synthetic and real-world data. This work opens several new research directions, including the development of more efficient SGD updates with consistent estimators and the design of efficient training algorithms for large-scale graphs. |
| Tasks | Stochastic Optimization |
| Published | 2018-07-31 |
| URL | https://arxiv.org/abs/1807.11880v4 |
| https://arxiv.org/pdf/1807.11880v4.pdf | |
| PWC | https://paperswithcode.com/paper/stochastic-gradient-descent-with-biased-but |
| Repo | https://github.com/jiechenjiechen/FastGCN-matlab |
| Framework | tf |
Maximizing Invariant Data Perturbation with Stochastic Optimization
| Title | Maximizing Invariant Data Perturbation with Stochastic Optimization |
| Authors | Kouichi Ikeno, Satoshi Hara |
| Abstract | Feature attribution methods, or saliency maps, are one of the most popular approaches for explaining the decisions of complex machine learning models such as deep neural networks. In this study, we propose a stochastic optimization approach for the perturbation-based feature attribution method. While the original optimization problem of the perturbation-based feature attribution is difficult to solve because of the complex constraints, we propose to reformulate the problem as the maximization of a differentiable function, which can be solved using gradient-based algorithms. In particular, stochastic optimization is well-suited for the proposed reformulation, and we can solve the problem using popular algorithms such as SGD, RMSProp, and Adam. The experiment on the image classification with VGG16 shows that the proposed method could identify relevant parts of the images effectively. |
| Tasks | Image Classification, Stochastic Optimization |
| Published | 2018-07-12 |
| URL | http://arxiv.org/abs/1807.05077v2 |
| http://arxiv.org/pdf/1807.05077v2.pdf | |
| PWC | https://paperswithcode.com/paper/maximizing-invariant-data-perturbation-with |
| Repo | https://github.com/sato9hara/PertMap |
| Framework | tf |
Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation
| Title | Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation |
| Authors | Md Zahangir Alom, Mahmudul Hasan, Chris Yakopcic, Tarek M. Taha, Vijayan K. Asari |
| Abstract | Deep learning (DL) based semantic segmentation methods have been providing state-of-the-art performance in the last few years. More specifically, these techniques have been successfully applied to medical image classification, segmentation, and detection tasks. One deep learning technique, U-Net, has become one of the most popular for these applications. In this paper, we propose a Recurrent Convolutional Neural Network (RCNN) based on U-Net as well as a Recurrent Residual Convolutional Neural Network (RRCNN) based on U-Net models, which are named RU-Net and R2U-Net respectively. The proposed models utilize the power of U-Net, Residual Network, as well as RCNN. There are several advantages of these proposed architectures for segmentation tasks. First, a residual unit helps when training deep architecture. Second, feature accumulation with recurrent residual convolutional layers ensures better feature representation for segmentation tasks. Third, it allows us to design better U-Net architecture with same number of network parameters with better performance for medical image segmentation. The proposed models are tested on three benchmark datasets such as blood vessel segmentation in retina images, skin cancer segmentation, and lung lesion segmentation. The experimental results show superior performance on segmentation tasks compared to equivalent models including U-Net and residual U-Net (ResU-Net). |
| Tasks | Image Classification, Lesion Segmentation, Lung Nodule Segmentation, Medical Image Segmentation, Retinal Vessel Segmentation, Semantic Segmentation, Skin Cancer Segmentation |
| Published | 2018-02-20 |
| URL | http://arxiv.org/abs/1802.06955v5 |
| http://arxiv.org/pdf/1802.06955v5.pdf | |
| PWC | https://paperswithcode.com/paper/recurrent-residual-convolutional-neural |
| Repo | https://github.com/1044197988/TF.Keras-Commonly-used-models |
| Framework | tf |
