Paper Group ANR 1633
Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Loss. Integrating Spatial Configuration into Heatmap Regression Based CNNs for Landmark Localization. Road Network Reconstruction from Satellite Images with Machine Learning Supported by Topological Methods. Composite Neural Network: Theory and Application to PM2 …
Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Loss
| Title | Improving the Robustness of Deep Neural Networks via Adversarial Training with Triplet Loss |
| Authors | Pengcheng Li, Jinfeng Yi, Bowen Zhou, Lijun Zhang |
| Abstract | Recent studies have highlighted that deep neural networks (DNNs) are vulnerable to adversarial examples. In this paper, we improve the robustness of DNNs by utilizing techniques of Distance Metric Learning. Specifically, we incorporate Triplet Loss, one of the most popular Distance Metric Learning methods, into the framework of adversarial training. Our proposed algorithm, Adversarial Training with Triplet Loss (AT$^2$L), substitutes the adversarial example against the current model for the anchor of triplet loss to effectively smooth the classification boundary. Furthermore, we propose an ensemble version of AT$^2$L, which aggregates different attack methods and model structures for better defense effects. Our empirical studies verify that the proposed approach can significantly improve the robustness of DNNs without sacrificing accuracy. Finally, we demonstrate that our specially designed triplet loss can also be used as a regularization term to enhance other defense methods. |
| Tasks | Metric Learning |
| Published | 2019-05-28 |
| URL | https://arxiv.org/abs/1905.11713v1 |
| https://arxiv.org/pdf/1905.11713v1.pdf | |
| PWC | https://paperswithcode.com/paper/improving-the-robustness-of-deep-neural-2 |
| Repo | |
| Framework | |
Integrating Spatial Configuration into Heatmap Regression Based CNNs for Landmark Localization
| Title | Integrating Spatial Configuration into Heatmap Regression Based CNNs for Landmark Localization |
| Authors | Christian Payer, Darko Štern, Horst Bischof, Martin Urschler |
| Abstract | In many medical image analysis applications, often only a limited amount of training data is available, which makes training of convolutional neural networks (CNNs) challenging. In this work on anatomical landmark localization, we propose a CNN architecture that learns to split the localization task into two simpler sub-problems, reducing the need for large training datasets. Our fully convolutional SpatialConfiguration-Net (SCN) dedicates one component to locally accurate but ambiguous candidate predictions, while the other component improves robustness to ambiguities by incorporating the spatial configuration of landmarks. In our experimental evaluation, we show that the proposed SCN outperforms related methods in terms of landmark localization error on size-limited datasets. |
| Tasks | |
| Published | 2019-08-02 |
| URL | https://arxiv.org/abs/1908.00748v1 |
| https://arxiv.org/pdf/1908.00748v1.pdf | |
| PWC | https://paperswithcode.com/paper/integrating-spatial-configuration-into |
| Repo | |
| Framework | |
Road Network Reconstruction from Satellite Images with Machine Learning Supported by Topological Methods
| Title | Road Network Reconstruction from Satellite Images with Machine Learning Supported by Topological Methods |
| Authors | Tamal K. Dey, Jiayuan Wang, Yusu Wang |
| Abstract | Automatic Extraction of road network from satellite images is a goal that can benefit and even enable new technologies. Methods that combine machine learning (ML) and computer vision have been proposed in recent years which make the task semi-automatic by requiring the user to provide curated training samples. The process can be fully automatized if training samples can be produced algorithmically. Of course, this requires a robust algorithm that can reconstruct the road networks from satellite images reliably so that the output can be fed as training samples. In this work, we develop such a technique by infusing a persistence-guided discrete Morse based graph reconstruction algorithm into ML framework. We elucidate our contributions in two phases. First, in a semi-automatic framework, we combine a discrete-Morse based graph reconstruction algorithm with an existing CNN framework to segment input satellite images. We show that this leads to reconstructions with better connectivity and less noise. Next, in a fully automatic framework, we leverage the power of the discrete-Morse based graph reconstruction algorithm to train a CNN from a collection of images without labelled data and use the same algorithm to produce the final output from the segmented images created by the trained CNN. We apply the discrete-Morse based graph reconstruction algorithm iteratively to improve the accuracy of the CNN. We show promising experimental results of this new framework on datasets from SpaceNet Challenge. |
| Tasks | |
| Published | 2019-09-15 |
| URL | https://arxiv.org/abs/1909.06728v1 |
| https://arxiv.org/pdf/1909.06728v1.pdf | |
| PWC | https://paperswithcode.com/paper/road-network-reconstruction-from-satellite |
| Repo | |
| Framework | |
Composite Neural Network: Theory and Application to PM2.5 Prediction
| Title | Composite Neural Network: Theory and Application to PM2.5 Prediction |
| Authors | Ming-Chuan Yang, Meng Chang Chen |
| Abstract | This work investigates the framework and performance issues of the composite neural network, which is composed of a collection of pre-trained and non-instantiated neural network models connected as a rooted directed acyclic graph for solving complicated applications. A pre-trained neural network model is generally well trained, targeted to approximate a specific function. Despite a general belief that a composite neural network may perform better than a single component, the overall performance characteristics are not clear. In this work, we construct the framework of a composite network, and prove that a composite neural network performs better than any of its pre-trained components with a high probability bound. In addition, if an extra pre-trained component is added to a composite network, with high probability, the overall performance will not be degraded. In the study, we explore a complicated application—PM2.5 prediction—to illustrate the correctness of the proposed composite network theory. In the empirical evaluations of PM2.5 prediction, the constructed composite neural network models support the proposed theory and perform better than other machine learning models, demonstrate the advantages of the proposed framework. |
| Tasks | |
| Published | 2019-10-22 |
| URL | https://arxiv.org/abs/1910.09739v1 |
| https://arxiv.org/pdf/1910.09739v1.pdf | |
| PWC | https://paperswithcode.com/paper/composite-neural-network-theory-and |
| Repo | |
| Framework | |
Multi-Temporal Aerial Image Registration Using Semantic Features
| Title | Multi-Temporal Aerial Image Registration Using Semantic Features |
| Authors | Ananya Gupta, Yao Peng, Simon Watson, Hujun Yin |
| Abstract | A semantic feature extraction method for multitemporal high resolution aerial image registration is proposed in this paper. These features encode properties or information about temporally invariant objects such as roads and help deal with issues such as changing foliage in image registration, which classical handcrafted features are unable to address. These features are extracted from a semantic segmentation network and have shown good robustness and accuracy in registering aerial images across years and seasons in the experiments. |
| Tasks | Image Registration, Semantic Segmentation |
| Published | 2019-08-30 |
| URL | https://arxiv.org/abs/1908.11822v2 |
| https://arxiv.org/pdf/1908.11822v2.pdf | |
| PWC | https://paperswithcode.com/paper/multi-temporal-high-resolution-aerial-image |
| Repo | |
| Framework | |
Virtual Thin Slice: 3D Conditional GAN-based Super-resolution for CT Slice Interval
| Title | Virtual Thin Slice: 3D Conditional GAN-based Super-resolution for CT Slice Interval |
| Authors | Akira Kudo, Yoshiro Kitamura, Yuanzhong Li, Satoshi Iizuka, Edgar Simo-Serra |
| Abstract | Many CT slice images are stored with large slice intervals to reduce storage size in clinical practice. This leads to low resolution perpendicular to the slice images (i.e., z-axis), which is insufficient for 3D visualization or image analysis. In this paper, we present a novel architecture based on conditional Generative Adversarial Networks (cGANs) with the goal of generating high resolution images of main body parts including head, chest, abdomen and legs. However, GANs are known to have a difficulty with generating a diversity of patterns due to a phenomena known as mode collapse. To overcome the lack of generated pattern variety, we propose to condition the discriminator on the different body parts. Furthermore, our generator networks are extended to be three dimensional fully convolutional neural networks, allowing for the generation of high resolution images from arbitrary fields of view. In our verification tests, we show that the proposed method obtains the best scores by PSNR/SSIM metrics and Visual Turing Test, allowing for accurate reproduction of the principle anatomy in high resolution. We expect that the proposed method contribute to effective utilization of the existing vast amounts of thick CT images stored in hospitals. |
| Tasks | Super-Resolution |
| Published | 2019-08-30 |
| URL | https://arxiv.org/abs/1908.11506v2 |
| https://arxiv.org/pdf/1908.11506v2.pdf | |
| PWC | https://paperswithcode.com/paper/virtual-thin-slice-3d-conditional-gan-based |
| Repo | |
| Framework | |
Dynamic Connected Neural Decision Classifier and Regressor with Dynamic Softing Pruning
| Title | Dynamic Connected Neural Decision Classifier and Regressor with Dynamic Softing Pruning |
| Authors | Faen Zhang, Xinyu Fan, Hui Xu, Pengcheng Zhou, Yujian He, Junlong Liu |
| Abstract | To deal with datasets of different complexity, this paper presents an efficient learning model that combines the proposed Dynamic Connected Neural Decision Networks (DNDN) and a new pruning method–Dynamic Soft Pruning (DSP). DNDN is a combination of random forests and deep neural networks thereby it enjoys both the properties of powerful classification capability and representation learning functionality. Different from Deep Neural Decision Forests (DNDF), this paper adopts an end-to-end training approach by representing the classification distribution with multiple randomly initialized softmax layers, which enables the placement of the forest trees after each layer in the neural network and greatly improves the training speed and stability. Furthermore, DSP is proposed to reduce the redundant connections of the network in a soft fashion which has high flexibility but demonstrates no performance loss compared with previous approaches. Extensive experiments on different datasets demonstrate the superiority of the proposed model over other popular algorithms in solving classification tasks. |
| Tasks | Representation Learning |
| Published | 2019-11-13 |
| URL | https://arxiv.org/abs/1911.05443v2 |
| https://arxiv.org/pdf/1911.05443v2.pdf | |
| PWC | https://paperswithcode.com/paper/dynamic-connected-neural-decision-classifier |
| Repo | |
| Framework | |
Dual Adversarial Semantics-Consistent Network for Generalized Zero-Shot Learning
| Title | Dual Adversarial Semantics-Consistent Network for Generalized Zero-Shot Learning |
| Authors | Jian Ni, Shanghang Zhang, Haiyong Xie |
| Abstract | Generalized zero-shot learning (GZSL) is a challenging class of vision and knowledge transfer problems in which both seen and unseen classes appear during testing. Existing GZSL approaches either suffer from semantic loss and discard discriminative information at the embedding stage, or cannot guarantee the visual-semantic interactions. To address these limitations, we propose the Dual Adversarial Semantics-Consistent Network (DASCN), which learns primal and dual Generative Adversarial Networks (GANs) in a unified framework for GZSL. In particular, the primal GAN learns to synthesize inter-class discriminative and semantics-preserving visual features from both the semantic representations of seen/unseen classes and the ones reconstructed by the dual GAN. The dual GAN enforces the synthetic visual features to represent prior semantic knowledge well via semantics-consistent adversarial learning. To the best of our knowledge, this is the first work that employs a novel dual-GAN mechanism for GZSL. Extensive experiments show that our approach achieves significant improvements over the state-of-the-art approaches. |
| Tasks | Transfer Learning, Zero-Shot Learning |
| Published | 2019-07-12 |
| URL | https://arxiv.org/abs/1907.05570v1 |
| https://arxiv.org/pdf/1907.05570v1.pdf | |
| PWC | https://paperswithcode.com/paper/dual-adversarial-semantics-consistent-network |
| Repo | |
| Framework | |
Deep Switch Networks for Generating Discrete Data and Language
| Title | Deep Switch Networks for Generating Discrete Data and Language |
| Authors | Payam Delgosha, Naveen Goela |
| Abstract | Multilayer switch networks are proposed as artificial generators of high-dimensional discrete data (e.g., binary vectors, categorical data, natural language, network log files, and discrete-valued time series). Unlike deconvolution networks which generate continuous-valued data and which consist of upsampling filters and reverse pooling layers, multilayer switch networks are composed of adaptive switches which model conditional distributions of discrete random variables. An interpretable, statistical framework is introduced for training these nonlinear networks based on a maximum-likelihood objective function. To learn network parameters, stochastic gradient descent is applied to the objective. This direct optimization is stable until convergence, and does not involve back-propagation over separate encoder and decoder networks, or adversarial training of dueling networks. While training remains tractable for moderately sized networks, Markov-chain Monte Carlo (MCMC) approximations of gradients are derived for deep networks which contain latent variables. The statistical framework is evaluated on synthetic data, high-dimensional binary data of handwritten digits, and web-crawled natural language data. Aspects of the model’s framework such as interpretability, computational complexity, and generalization ability are discussed. |
| Tasks | Time Series |
| Published | 2019-03-14 |
| URL | http://arxiv.org/abs/1903.06135v1 |
| http://arxiv.org/pdf/1903.06135v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-switch-networks-for-generating-discrete |
| Repo | |
| Framework | |
Bayesian Optimization Meets Riemannian Manifolds in Robot Learning
| Title | Bayesian Optimization Meets Riemannian Manifolds in Robot Learning |
| Authors | Noémie Jaquier, Leonel Rozo, Sylvain Calinon, Mathias Bürger |
| Abstract | Bayesian optimization (BO) recently became popular in robotics to optimize control parameters and parametric policies in direct reinforcement learning due to its data efficiency and gradient-free approach. However, its performance may be seriously compromised when the parameter space is high-dimensional. A way to tackle this problem is to introduce domain knowledge into the BO framework. We propose to exploit the geometry of non-Euclidean parameter spaces, which often arise in robotics (e.g. orientation, stiffness matrix). Our approach, built on Riemannian manifold theory, allows BO to properly measure similarities in the parameter space through geometry-aware kernel functions and to optimize the acquisition function on the manifold as an unconstrained problem. We test our approach in several benchmark artificial landscapes and using a 7-DOF simulated robot to learn orientation and impedance parameters for manipulation skills. |
| Tasks | |
| Published | 2019-10-11 |
| URL | https://arxiv.org/abs/1910.04998v1 |
| https://arxiv.org/pdf/1910.04998v1.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-optimization-meets-riemannian |
| Repo | |
| Framework | |
Distribution-Independent PAC Learning of Halfspaces with Massart Noise
| Title | Distribution-Independent PAC Learning of Halfspaces with Massart Noise |
| Authors | Ilias Diakonikolas, Themis Gouleakis, Christos Tzamos |
| Abstract | We study the problem of {\em distribution-independent} PAC learning of halfspaces in the presence of Massart noise. Specifically, we are given a set of labeled examples $(\mathbf{x}, y)$ drawn from a distribution $\mathcal{D}$ on $\mathbb{R}^{d+1}$ such that the marginal distribution on the unlabeled points $\mathbf{x}$ is arbitrary and the labels $y$ are generated by an unknown halfspace corrupted with Massart noise at noise rate $\eta<1/2$. The goal is to find a hypothesis $h$ that minimizes the misclassification error $\mathbf{Pr}_{(\mathbf{x}, y) \sim \mathcal{D}} \left[ h(\mathbf{x}) \neq y \right]$. We give a $\mathrm{poly}\left(d, 1/\epsilon \right)$ time algorithm for this problem with misclassification error $\eta+\epsilon$. We also provide evidence that improving on the error guarantee of our algorithm might be computationally hard. Prior to our work, no efficient weak (distribution-independent) learner was known in this model, even for the class of disjunctions. The existence of such an algorithm for halfspaces (or even disjunctions) has been posed as an open question in various works, starting with Sloan (1988), Cohen (1997), and was most recently highlighted in Avrim Blum’s FOCS 2003 tutorial. |
| Tasks | |
| Published | 2019-06-24 |
| URL | https://arxiv.org/abs/1906.10075v2 |
| https://arxiv.org/pdf/1906.10075v2.pdf | |
| PWC | https://paperswithcode.com/paper/distribution-independent-pac-learning-of |
| Repo | |
| Framework | |
Evolving embodied intelligence from materials to machines
| Title | Evolving embodied intelligence from materials to machines |
| Authors | David Howard, Agoston E. Eiben, Danielle Frances Kennedy, Jean-Baptiste Mouret, Philip Valencia, Dave Winkler |
| Abstract | Natural lifeforms specialise to their environmental niches across many levels; from low-level features such as DNA and proteins, through to higher-level artefacts including eyes, limbs, and overarching body plans. We propose Multi-Level Evolution (MLE), a bottom-up automatic process that designs robots across multiple levels and niches them to tasks and environmental conditions. MLE concurrently explores constituent molecular and material ‘building blocks’, as well as their possible assemblies into specialised morphological and sensorimotor configurations. MLE provides a route to fully harness a recent explosion in available candidate materials and ongoing advances in rapid manufacturing processes. We outline a feasible MLE architecture that realises this vision, highlight the main roadblocks and how they may be overcome, and show robotic applications to which MLE is particularly suited. By forming a research agenda to stimulate discussion between researchers in related fields, we hope to inspire the pursuit of multi-level robotic design all the way from material to machine. |
| Tasks | |
| Published | 2019-01-17 |
| URL | http://arxiv.org/abs/1901.05704v1 |
| http://arxiv.org/pdf/1901.05704v1.pdf | |
| PWC | https://paperswithcode.com/paper/evolving-embodied-intelligence-from-materials |
| Repo | |
| Framework | |
Spatio-Temporal Segmentation in 3D Echocardiographic Sequences using Fractional Brownian Motion
| Title | Spatio-Temporal Segmentation in 3D Echocardiographic Sequences using Fractional Brownian Motion |
| Authors | Omar S. Al-Kadi |
| Abstract | An important aspect for an improved cardiac functional analysis is the accurate segmentation of the left ventricle (LV). A novel approach for fully-automated segmentation of the LV endocardium and epicardium contours is presented. This is mainly based on the natural physical characteristics of the LV shape structure. Both sides of the LV boundaries exhibit natural elliptical curvatures by having details on various scales, i.e. exhibiting fractal-like characteristics. The fractional Brownian motion (fBm), which is a non-stationary stochastic process, integrates well with the stochastic nature of ultrasound echoes. It has the advantage of representing a wide range of non-stationary signals and can quantify statistical local self-similarity throughout the time-sequence ultrasound images. The locally characterized boundaries of the fBm segmented LV were further iteratively refined using global information by means of second-order moments. The method is benchmarked using synthetic 3D+time echocardiographic sequences for normal and different ischemic cardiomyopathy, and results compared with state-of-the-art LV segmentation. Furthermore, the framework was validated against real data from canine cases with expert-defined segmentations and demonstrated improved accuracy. The fBm-based segmentation algorithm is fully automatic and has the potential to be used clinically together with 3D echocardiography for improved cardiovascular disease diagnosis. |
| Tasks | |
| Published | 2019-12-21 |
| URL | https://arxiv.org/abs/1912.10220v1 |
| https://arxiv.org/pdf/1912.10220v1.pdf | |
| PWC | https://paperswithcode.com/paper/spatio-temporal-segmentation-in-3d |
| Repo | |
| Framework | |
Adaptive Learning Expert System for Diagnosis and Management of Viral Hepatitis
| Title | Adaptive Learning Expert System for Diagnosis and Management of Viral Hepatitis |
| Authors | Henok Yared Agizew |
| Abstract | Viral hepatitis is the regularly found health problem throughout the world among other easily transmitted diseases, such as tuberculosis, human immune virus, malaria and so on. Among all hepatitis viruses, the uppermost numbers of deaths are result from the long-lasting hepatitis C infection or long-lasting hepatitis B. In order to develop this system, the knowledge is acquired using both structured and semi-structured interviews from internists of St.Paul Hospital. Once the knowledge is acquired, it is modeled and represented using rule based reasoning techniques. Both forward and backward chaining is used to infer the rules and provide appropriate advices in the developed expert system. For the purpose of developing the prototype expert system SWI-prolog editor also used. The proposed system has the ability to adapt with dynamic knowledge by generalizing rules and discover new rules through learning the newly arrived knowledge from domain experts adaptively without any help from the knowledge engineer. Keywords: Expert System, Diagnosis and Management of Viral Hepatitis, Adaptive Learning, Discovery and Generalization Mechanism |
| Tasks | |
| Published | 2019-04-09 |
| URL | http://arxiv.org/abs/1904.04937v1 |
| http://arxiv.org/pdf/1904.04937v1.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-learning-expert-system-for-diagnosis |
| Repo | |
| Framework | |
Temporal Wasserstein non-negative matrix factorization for non-rigid motion segmentation and spatiotemporal deconvolution
| Title | Temporal Wasserstein non-negative matrix factorization for non-rigid motion segmentation and spatiotemporal deconvolution |
| Authors | Erdem Varol, Amin Nejatbakhsh, Conor McGrory |
| Abstract | Motion segmentation for natural images commonly relies on dense optic flow to yield point trajectories which can be grouped into clusters through various means including spectral clustering or minimum cost multicuts. However, in biological imaging scenarios, such as fluorescence microscopy or calcium imaging, where the signal to noise ratio is compromised and intensity fluctuations occur, optical flow may be difficult to approximate. To this end, we propose an alternative paradigm for motion segmentation based on optimal transport which models the video frames as time-varying mass represented as histograms. Thus, we cast motion segmentation as a temporal non-linear matrix factorization problem with Wasserstein metric loss. The dictionary elements of this factorization yield segmentation of motion into coherent objects while the loading coefficients allow for time-varying intensity signal of the moving objects to be captured. We demonstrate the use of the proposed paradigm on a simulated multielectrode drift scenario, as well as calcium indicating fluorescence microscopy videos of the nematode Caenorhabditis elegans (C. elegans). The latter application has the added utility of extracting neural activity of the animal in freely conducted behavior. |
| Tasks | Motion Segmentation, Optical Flow Estimation |
| Published | 2019-12-07 |
| URL | https://arxiv.org/abs/1912.03463v1 |
| https://arxiv.org/pdf/1912.03463v1.pdf | |
| PWC | https://paperswithcode.com/paper/temporal-wasserstein-non-negative-matrix |
| Repo | |
| Framework | |
