October 16, 2019

2977 words 14 mins read

Paper Group ANR 1085

Style Decomposition for Improved Neural Style Transfer. RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars. Large Scale Audio-Visual Video Analytics Platform for Forensic Investigations of Terroristic Attacks. 3D Consistent Biventricular Myocardial Segmentation Using Deep Learning for Mesh Generation. Cross-Modality Synthe …

Style Decomposition for Improved Neural Style Transfer


Title	Style Decomposition for Improved Neural Style Transfer
Authors	Paraskevas Pegios, Nikolaos Passalis, Anastasios Tefas
Abstract	Universal Neural Style Transfer (NST) methods are capable of performing style transfer of arbitrary styles in a style-agnostic manner via feature transforms in (almost) real-time. Even though their unimodal parametric style modeling approach has been proven adequate to transfer a single style from relatively simple images, they are usually not capable of effectively handling more complex styles, producing significant artifacts, as well as reducing the quality of the synthesized textures in the stylized image. To overcome these limitations, in this paper we propose a novel universal NST approach that separately models each sub-style that exists in a given style image (or a collection of style images). This allows for better modeling the subtle style differences within the same style image and then using the most appropriate sub-style (or mixtures of different sub-styles) to stylize the content image. The ability of the proposed approach to a) perform a wide range of different stylizations using the sub-styles that exist in one style image, while giving the ability to the user to appropriate mix the different sub-styles, b) automatically match the most appropriate sub-style to different semantic regions of the content image, improving existing state-of-the-art universal NST approaches, and c) detecting and transferring the sub-styles from collections of images are demonstrated through extensive experiments.
Tasks	Style Transfer
Published	2018-11-30
URL	http://arxiv.org/abs/1811.12704v1
PDF	http://arxiv.org/pdf/1811.12704v1.pdf
PWC	https://paperswithcode.com/paper/style-decomposition-for-improved-neural-style
Repo
Framework

RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars


Title	RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars
Authors	Shubham Jain, Abhronil Sengupta, Kaushik Roy, Anand Raghunathan
Abstract	Resistive crossbars have emerged as promising building blocks for realizing DNNs due to their ability to compactly and efficiently realize the dominant DNN computational kernel, viz., vector-matrix multiplication. However, a key challenge with resistive crossbars is that they suffer from a range of device and circuit level non-idealities such as interconnect parasitics, peripheral circuits, sneak paths, and process variations. These non-idealities can lead to errors in vector-matrix multiplication that eventually degrade the DNN’s accuracy. There has been no study of the impact of non-idealities on the accuracy of large-scale DNNs, in part because existing device and circuit models are infeasible to use in application-level evaluation. In this work, we present a fast and accurate simulation framework to enable evaluation and re-training of large-scale DNNs on resistive crossbar based hardware fabrics. We first characterize the impact of crossbar non-idealities on errors incurred in the realized vector-matrix multiplications and observe that the errors have significant data and hardware-instance dependence that should be considered. We propose a Fast Crossbar Model (FCM) to accurately capture the errors arising due to crossbar non-idealities while being four-to-five orders of magnitude faster than circuit simulation. Finally, we develop RxNN, a software framework to evaluate and re-train DNNs on resistive crossbar systems. RxNN is based on the popular Caffe machine learning framework, and we use it to evaluate a suite of large-scale DNNs developed for the ImageNet Challenge (ILSVRC). Our experiments reveal that resistive crossbar non-idealities can lead to significant accuracy degradations (9.6%-32%) for these large-scale DNNs. To the best of our knowledge, this work is the first quantitative evaluation of the accuracy of large-scale DNNs on resistive crossbar based hardware.
Tasks
Published	2018-08-31
URL	http://arxiv.org/abs/1809.00072v2
PDF	http://arxiv.org/pdf/1809.00072v2.pdf
PWC	https://paperswithcode.com/paper/rxnn-a-framework-for-evaluating-deep-neural
Repo
Framework

Large Scale Audio-Visual Video Analytics Platform for Forensic Investigations of Terroristic Attacks


Title	Large Scale Audio-Visual Video Analytics Platform for Forensic Investigations of Terroristic Attacks
Authors	Alexander Schindler, Martin Boyer, Andrew Lindley, David Schreiber, Thomas Philipp
Abstract	The forensic investigation of a terrorist attack poses a huge challenge to the investigative authorities, as several thousand hours of video footage need to be spotted. To assist law enforcement agencies (LEA) in identifying suspects and securing evidences, we present a platform which fuses information of surveillance cameras and video uploads from eyewitnesses. The platform integrates analytical modules for different input-modalities on a scalable architecture. Videos are analyzed according their acoustic and visual content. Specifically, Audio Event Detection is applied to index the content according to attack-specific acoustic concepts. Audio similarity search is utilized to identify similar video sequences recorded from different perspectives. Visual object detection and tracking are used to index the content according to relevant concepts. The heterogeneous results of the analytical modules are fused into a distributed index of visual and acoustic concepts to facilitate rapid start of investigations, following traits and investigating witness reports.
Tasks	Object Detection
Published	2018-11-28
URL	http://arxiv.org/abs/1811.11623v1
PDF	http://arxiv.org/pdf/1811.11623v1.pdf
PWC	https://paperswithcode.com/paper/large-scale-audio-visual-video-analytics
Repo
Framework

3D Consistent Biventricular Myocardial Segmentation Using Deep Learning for Mesh Generation


Title	3D Consistent Biventricular Myocardial Segmentation Using Deep Learning for Mesh Generation
Authors	Qiao Zheng, Hervé Delingette, Nicolas Duchateau, Nicholas Ayache
Abstract	We present a novel automated method to segment the myocardium of both left and right ventricles in MRI volumes. The segmentation is consistent in 3D across the slices such that it can be directly used for mesh generation. Two specific neural networks with multi-scale coarse-to-fine prediction structure are proposed to cope with the small training dataset and trained using an original loss function. The former segments a slice in the middle of the volume. Then the latter iteratively propagates the slice segmentations towards the base and the apex, in a spatially consistent way. We perform 5-fold cross-validation on the 15 cases from STACOM to validate the method. For training, we use real cases and their synthetic variants generated by combining motion simulation and image synthesis. Accurate and consistent testing results are obtained.
Tasks	Image Generation
Published	2018-03-29
URL	http://arxiv.org/abs/1803.11080v1
PDF	http://arxiv.org/pdf/1803.11080v1.pdf
PWC	https://paperswithcode.com/paper/3d-consistent-biventricular-myocardial
Repo
Framework

Cross-Modality Synthesis from CT to PET using FCN and GAN Networks for Improved Automated Lesion Detection


Title	Cross-Modality Synthesis from CT to PET using FCN and GAN Networks for Improved Automated Lesion Detection
Authors	Avi Ben-Cohen, Eyal Klang, Stephen P. Raskin, Shelly Soffer, Simona Ben-Haim, Eli Konen, Michal Marianne Amitai, Hayit Greenspan
Abstract	In this work we present a novel system for generation of virtual PET images using CT scans. We combine a fully convolutional network (FCN) with a conditional generative adversarial network (GAN) to generate simulated PET data from given input CT data. The synthesized PET can be used for false-positive reduction in lesion detection solutions. Clinically, such solutions may enable lesion detection and drug treatment evaluation in a CT-only environment, thus reducing the need for the more expensive and radioactive PET/CT scan. Our dataset includes 60 PET/CT scans from Sheba Medical center. We used 23 scans for training and 37 for testing. Different schemes to achieve the synthesized output were qualitatively compared. Quantitative evaluation was conducted using an existing lesion detection software, combining the synthesized PET as a false positive reduction layer for the detection of malignant lesions in the liver. Current results look promising showing a 28% reduction in the average false positive per case from 2.9 to 2.1. The suggested solution is comprehensive and can be expanded to additional body organs, and different modalities.
Tasks
Published	2018-02-21
URL	http://arxiv.org/abs/1802.07846v2
PDF	http://arxiv.org/pdf/1802.07846v2.pdf
PWC	https://paperswithcode.com/paper/cross-modality-synthesis-from-ct-to-pet-using
Repo
Framework

Use of symmetric kernels for convolutional neural networks


Title	Use of symmetric kernels for convolutional neural networks
Authors	Viacheslav Dudar, Vladimir Semenov
Abstract	At this work we introduce horizontally symmetric convolutional kernels for CNNs which make the network output invariant to horizontal flips of the image. We also study other types of symmetric kernels which lead to vertical flip invariance, and approximate rotational invariance. We show that usage of such kernels acts as regularizer, and improves generalization of the convolutional neural networks at the cost of more complicated training process.
Tasks
Published	2018-05-23
URL	http://arxiv.org/abs/1805.09421v1
PDF	http://arxiv.org/pdf/1805.09421v1.pdf
PWC	https://paperswithcode.com/paper/use-of-symmetric-kernels-for-convolutional
Repo
Framework

Stovepiping and Malicious Software: A Critical Review of AGI Containment


Title	Stovepiping and Malicious Software: A Critical Review of AGI Containment
Authors	Jason M. Pittman, Jesus P. Espinoza, Courtney Soboleski Crosby
Abstract	Awareness of the possible impacts associated with artificial intelligence has risen in proportion to progress in the field. While there are tremendous benefits to society, many argue that there are just as many, if not more, concerns related to advanced forms of artificial intelligence. Accordingly, research into methods to develop artificial intelligence safely is increasingly important. In this paper, we provide an overview of one such safety paradigm: containment with a critical lens aimed toward generative adversarial networks and potentially malicious artificial intelligence. Additionally, we illuminate the potential for a developmental blindspot in the stovepiping of containment mechanisms.
Tasks
Published	2018-11-08
URL	http://arxiv.org/abs/1811.03653v1
PDF	http://arxiv.org/pdf/1811.03653v1.pdf
PWC	https://paperswithcode.com/paper/stovepiping-and-malicious-software-a-critical
Repo
Framework

Initialization of ReLUs for Dynamical Isometry


Title	Initialization of ReLUs for Dynamical Isometry
Authors	Rebekka Burkholz, Alina Dubatovka
Abstract	Deep learning relies on good initialization schemes and hyperparameter choices prior to training a neural network. Random weight initializations induce random network ensembles, which give rise to the trainability, training speed, and sometimes also generalization ability of an instance. In addition, such ensembles provide theoretical insights into the space of candidate models of which one is selected during training. The results obtained so far rely on mean field approximations that assume infinite layer width and that study average squared signals. We derive the joint signal output distribution exactly, without mean field assumptions, for fully-connected networks with Gaussian weights and biases, and analyze deviations from the mean field results. For rectified linear units, we further discuss limitations of the standard initialization scheme, such as its lack of dynamical isometry, and propose a simple alternative that overcomes these by initial parameter sharing.
Tasks
Published	2018-06-17
URL	https://arxiv.org/abs/1806.06362v3
PDF	https://arxiv.org/pdf/1806.06362v3.pdf
PWC	https://paperswithcode.com/paper/exact-information-propagation-through-fully
Repo
Framework

Sample Reuse via Importance Sampling in Information Geometric Optimization


Title	Sample Reuse via Importance Sampling in Information Geometric Optimization
Authors	Shinichi Shirakawa, Youhei Akimoto, Kazuki Ouchi, Kouzou Ohara
Abstract	In this paper we propose a technique to reduce the number of function evaluations, which is often the bottleneck of the black-box optimization, in the information geometric optimization (IGO) that is a generic framework of the probability model-based black-box optimization algorithms and generalizes several well-known evolutionary algorithms, such as the population-based incremental learning (PBIL) and the pure rank-$\mu$ update covariance matrix adaptation evolution strategy (CMA-ES). In each iteration, the IGO algorithms update the parameters of the probability distribution to the natural gradient direction estimated by Monte-Carlo with the samples drawn from the current distribution. Our strategy is to reuse previously generated and evaluated samples based on the importance sampling. It is a technique to reduce the estimation variance without introducing a bias in Monte-Carlo estimation. We apply the sample reuse technique to the PBIL and the pure rank-$\mu$ update CMA-ES and empirically investigate its effect. The experimental results show that the sample reuse helps to reduce the number of function evaluations on many benchmark functions for both the PBIL and the pure rank-$\mu$ update CMA-ES. Moreover, we demonstrate how to combine the importance sampling technique with a variant of the CMA-ES involving an algorithmic component that is not derived in the IGO framework.
Tasks
Published	2018-05-31
URL	http://arxiv.org/abs/1805.12388v1
PDF	http://arxiv.org/pdf/1805.12388v1.pdf
PWC	https://paperswithcode.com/paper/sample-reuse-via-importance-sampling-in
Repo
Framework

Unsupervised Learning using Pretrained CNN and Associative Memory Bank


Title	Unsupervised Learning using Pretrained CNN and Associative Memory Bank
Authors	Qun Liu, Supratik Mukhopadhyay
Abstract	Deep Convolutional features extracted from a comprehensive labeled dataset, contain substantial representations which could be effectively used in a new domain. Despite the fact that generic features achieved good results in many visual tasks, fine-tuning is required for pretrained deep CNN models to be more effective and provide state-of-the-art performance. Fine tuning using the backpropagation algorithm in a supervised setting, is a time and resource consuming process. In this paper, we present a new architecture and an approach for unsupervised object recognition that addresses the above mentioned problem with fine tuning associated with pretrained CNN-based supervised deep learning approaches while allowing automated feature extraction. Unlike existing works, our approach is applicable to general object recognition tasks. It uses a pretrained (on a related domain) CNN model for automated feature extraction pipelined with a Hopfield network based associative memory bank for storing patterns for classification purposes. The use of associative memory bank in our framework allows eliminating backpropagation while providing competitive performance on an unseen dataset.
Tasks	Object Recognition
Published	2018-05-02
URL	http://arxiv.org/abs/1805.01033v1
PDF	http://arxiv.org/pdf/1805.01033v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-learning-using-pretrained-cnn
Repo
Framework

Improved Training of Generative Adversarial Networks Using Representative Features


Title	Improved Training of Generative Adversarial Networks Using Representative Features
Authors	Duhyeon Bang, Hyunjung Shim
Abstract	Despite the success of generative adversarial networks (GANs) for image generation, the trade-off between visual quality and image diversity remains a significant issue. This paper achieves both aims simultaneously by improving the stability of training GANs. The key idea of the proposed approach is to implicitly regularize the discriminator using representative features. Focusing on the fact that standard GAN minimizes reverse Kullback-Leibler (KL) divergence, we transfer the representative feature, which is extracted from the data distribution using a pre-trained autoencoder (AE), to the discriminator of standard GANs. Because the AE learns to minimize forward KL divergence, our GAN training with representative features is influenced by both reverse and forward KL divergence. Consequently, the proposed approach is verified to improve visual quality and diversity of state of the art GANs using extensive evaluations.
Tasks	Image Generation
Published	2018-01-28
URL	http://arxiv.org/abs/1801.09195v3
PDF	http://arxiv.org/pdf/1801.09195v3.pdf
PWC	https://paperswithcode.com/paper/improved-training-of-generative-adversarial
Repo
Framework

Prediction of Industrial Process Parameters using Artificial Intelligence Algorithms


Title	Prediction of Industrial Process Parameters using Artificial Intelligence Algorithms
Authors	Abdelmoula Khdoudi, Tawfik Masrour
Abstract	In the present paper, a method of defining the industrial process parameters for a new product using machine learning algorithms will be presented. The study will describe how to go from the product characteristics till the prediction of the suitable machine parameters to produce a good quality of this product, and this is based on an historical training dataset of similar products with their respective process parameters. In the first part of our study, we will focus on the ultrasonic welding process definition, welding parameters and on how it operate. While in second part, we present the design and implementation of the prediction models such multiple linear regression, support vector regression, and we compare them to an artificial neural networks algorithm. In the following part, we present a new application of Convolutional Neural Networks (CNN) to the industrial process parameters prediction. In addition, we will propose the generalization approach of our CNN to any prediction problem of industrial process parameters. Finally the results of the four methods will be interpreted and discussed.
Tasks
Published	2018-12-26
URL	http://arxiv.org/abs/1812.10537v2
PDF	http://arxiv.org/pdf/1812.10537v2.pdf
PWC	https://paperswithcode.com/paper/prediction-of-industrial-process-parameters
Repo
Framework

Value-aware Quantization for Training and Inference of Neural Networks


Title	Value-aware Quantization for Training and Inference of Neural Networks
Authors	Eunhyeok Park, Sungjoo Yoo, Peter Vajda
Abstract	We propose a novel value-aware quantization which applies aggressively reduced precision to the majority of data while separately handling a small amount of large data in high precision, which reduces total quantization errors under very low precision. We present new techniques to apply the proposed quantization to training and inference. The experiments show that our method with 3-bit activations (with 2% of large ones) can give the same training accuracy as full-precision one while offering significant (41.6% and 53.7%) reductions in the memory cost of activations in ResNet-152 and Inception-v3 compared with the state-of-the-art method. Our experiments also show that deep networks such as Inception-v3, ResNet-101 and DenseNet-121 can be quantized for inference with 4-bit weights and activations (with 1% 16-bit data) within 1% top-1 accuracy drop.
Tasks	Quantization
Published	2018-04-20
URL	http://arxiv.org/abs/1804.07802v1
PDF	http://arxiv.org/pdf/1804.07802v1.pdf
PWC	https://paperswithcode.com/paper/value-aware-quantization-for-training-and
Repo
Framework

Statistical mechanics of low-rank tensor decomposition


Title	Statistical mechanics of low-rank tensor decomposition
Authors	Jonathan Kadmon, Surya Ganguli
Abstract	Often, large, high dimensional datasets collected across multiple modalities can be organized as a higher order tensor. Low-rank tensor decomposition then arises as a powerful and widely used tool to discover simple low dimensional structures underlying such data. However, we currently lack a theoretical understanding of the algorithmic behavior of low-rank tensor decompositions. We derive Bayesian approximate message passing (AMP) algorithms for recovering arbitrarily shaped low-rank tensors buried within noise, and we employ dynamic mean field theory to precisely characterize their performance. Our theory reveals the existence of phase transitions between easy, hard and impossible inference regimes, and displays an excellent match with simulations. Moreover, it reveals several qualitative surprises compared to the behavior of symmetric, cubic tensor decomposition. Finally, we compare our AMP algorithm to the most commonly used algorithm, alternating least squares (ALS), and demonstrate that AMP significantly outperforms ALS in the presence of noise.
Tasks
Published	2018-10-23
URL	http://arxiv.org/abs/1810.10065v1
PDF	http://arxiv.org/pdf/1810.10065v1.pdf
PWC	https://paperswithcode.com/paper/statistical-mechanics-of-low-rank-tensor
Repo
Framework

A recurrent multi-scale approach to RBG-D Object Recognition


Title	A recurrent multi-scale approach to RBG-D Object Recognition
Authors	Mirco Planamente, Mohammad Reza Loghmani, Barbara Caputo
Abstract	Technological development aims to produce generations of increasingly efficient robots able to perform complex tasks. This requires considerable efforts, from the scientific community, to find new algorithms that solve computer vision problems, such as object recognition. The diffusion of RGB-D cameras directed the study towards the research of new architectures able to exploit the RGB and Depth information. The project that is developed in this thesis concerns the realization of a new end-to-end architecture for the recognition of RGB-D objects called RCFusion. Our method generates compact and highly discriminative multi-modal features by combining complementary RGB and depth information representing different levels of abstraction. We evaluate our method on standard object recognition datasets, RGB-D Object Dataset and JHUIT-50. The experiments performed show that our method outperforms the existing approaches and establishes new state-of-the-art results for both datasets.
Tasks	Object Recognition
Published	2018-07-31
URL	http://arxiv.org/abs/1808.01357v3
PDF	http://arxiv.org/pdf/1808.01357v3.pdf
PWC	https://paperswithcode.com/paper/a-recurrent-multi-scale-approach-to-rbg-d
Repo
Framework